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ABSTRACT 

This  is  a  study  of  the  state-of-the-art  in  Visual  Analytics  (VA)  technologies  that  might 
support  the  maintenance  of  the  Recognized  Maritime  Picture  (RMP).  It  begins  with  an 
analysis  of  the  term  “Visual  Analytics,”  including  what  qualifies  and  does  not  qualify 
under  that  term,  and  proposes  a  4-point  checklist  of  required  elements.  It  reviews  what 
interaction  patterns  are  typical  in  VA  and  what  topics  need  to  be  visualized  for  the 
RMP,  and  thus  establishes  a  list  of  required  tool  capabilities.  Links  are  then  provided  to 
approximately  70  papers  that  address  these  requirements.  Global  VA  researchers, 
research  groups,  journals,  and  conferences  are  summarized.  Fifteen  VA  products  that 
are  particularly  relevant  to  the  identified  requirements  are  tabulated.  An  annotated 
bibliography,  attached  as  an  appendix,  lists  and  provides  an  educated  assessment  of  146 
relevant  papers,  websites,  and  product  brochures. 

RESUME 


Ceci  est  une  etude  de  l’etat  de  Part  des  technologies  d’Analytique  Visuelle  (AV)  qui 
seraient  susceptibles  de  soutenir  la  mise  a  jour  du  Tableau  de  la  Situation  Maritime 
(TSM).  Elle  debute  avec  une  analyse  du  terme  «  Analytique  Visuelle  »,  incluant  ce  qui 
se  qualifie  et  ne  se  qualifie  pas  sous  cette  definition,  et  propose  une  liste  de  controle  de 
quatre  points  des  elements  requis.  Elle  passe  en  revue  les  modeles  d’ interaction 
typiques  en  AV  et  les  sujets  qui  doivent  etre  visualises  pour  le  TSM  et  etablit  ainsi  une 
liste  des  fonctionalites  requises  pour  l’outil.  Ensuite,  des  liens  vers  environ  70  articles 
qui  addressent  ces  exigences  sont  foumis.  Un  resume  des  chercheurs  en  AV,  des 
groupes  de  recherche,  des  journaux  et  des  conferences  est  aussi  donne.  Quinze  produits 
d’Analytique  Visuelle  qui  sont  particulierement  pertinents  en  regard  des  exigences 
identifies  sont  colliges.  Finalement,  une  bibliographic  anno  tee  enumerant  et 
fournissant  une  evaluation  de  146  articles,  sites  internet  et  brochures  de  produits  se 
trouve  en  annexe. 
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EXECUTIVE  SUMMARY 

This  report  describes  a  study  of  state-of-the-art  Visual  Analytics  (VA)  approaches  and 
technologies  that  could  be  used  to  improve  the  development  of,  and  interaction  with,  the 
Recognized  Maritime  Picture  (RMP).  Given  a  tendency  for  the  phrase  “Visual 
Analytics”  to  be  used  quite  liberally,  Section  2  first  examines  what  the  tenn  means  and 
does  not  mean  and  concludes  that,  to  be  VA,  a  tool  must  include: 

•  Visualization:  Use  complex  and  abstract  depictions 

•  Interaction:  Allow  rich  interactions  with  the  visualization 

•  Exploration:  Facilitate  the  discovery  of  new  information 

•  Analysis:  Encourage  users  to  assemble  evidence  into  a  “story” 

Section  3  then  characterizes  the  global  VA  research  community  by  introducing  nine  key 
researchers  and  technology  leaders  and  the  institutions  that  they  work  with.  This 
includes  an  overview  of  the  Visual  Analytics  Consortium  in  North  America,  and 
parallel  developments  in  Europe. 

Section  4  is  the  main  Literature  Review.  The  first  subsection  lists  nine  journals  and 
conferences  that  serve  the  VA  research  community,  noting  in  particular  the  importance 
of  IEEE’s  Visual  Analytics  Science  and  Technology  (VAST)  conference.  Because 
many  important  papers  are  self-published  or  released  as  preprints  on  the  web, 

Section  4.1.4  also  lists  those  research  institutes  that  maintain  home  pages  with 
important  VA  content.  Based  on  separate  consultations  with  RMP  experts,  Section  4.2 
identifies  the  following  VA  capabilities  that  are  of  particular  interest: 

•  Geo-spatial-temporal  situation  awareness 

•  Geo-spatial-temporal  data  mining 

•  Geo-spatial-temporal  uncertainty  representation 

•  Geo-temporal  coverage  analysis 

•  Predictions  and  what-if  analysis 

•  Sensor  and  network  monitoring 

•  Social  network  analysis 

•  Collaboration 

•  Textual  and  news-feed  exploration 

•  User  interfaces  and  usability  for  Visual  Analytics 

•  Visual  Analytics  design  principles 

Section  4.3  identifies  and  briefly  introduces  approximately  70  publications  that  address 
these  topics. 


xii 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


MDA 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


Section  5  then  reviews  fifteen  VA  products  that  are  already  available  with  some  or  all 
of  these  capabilities.  A  table  for  each  product  provides  a  screen-grab  or  image,  a  brief 
review  of  its  capabilities  and  limitations,  an  assessment  of  whether  it  really  is  Visual 
Analytics  (based  on  the  above  four  requirements),  and  links  to  further  infonnation. 
Video  clips  are  attached  where  possible,  because  they  provide  a  better  description  of 
user- interactions  that  are  very  important  for  Visual  Analytics. 

Appendix  B  is  an  annotated  bibliography  of  the  papers,  websites,  and  products  cited  in 
this  paper.  Each  listing  provides  standard  citation  information,  web  links  if  available, 
and  an  infonned  assessment  of  the  relevance  and  value  of  the  item  cited.  Where 
possible,  copies  of  the  cited  documents  and  videos  have  been  delivered  as  digital  files 
with  this  report,  and  those  files  are  referenced  by  name  in  the  bibliography.  The 
EndNote  database  used  to  generate  the  bibliography  has  also  been  delivered. 

RAPPORT  SOMMAIRE 

Ce  rapport  decrit  une  etude  sur  l’etat  de  l’art  des  technologies  et  approches  en 
Analytique  Visuelle  (AV)  qui  pourraient  etre  utilisees  pour  ameliorer  le  developpement 
du  Tableau  de  la  Situation  Maritime  (TSM)  et  Tinteraction  avec  ce  dernier.  Considerant 
le  fait  que  T  expression  «  Analytique  Visuelle  »  est  employee  assez  liberalement,  la 
Section  2  examine  ce  que  Texpression  signifie  et  ne  signifie  pas  et  conclut  que,  pour 
etre  considere  AV,  un  outil  doit  comprendre  : 

•  Visualisation  :  Employer  des  representations  complexes  et  abstraites 

•  Interaction  :  Permettre  des  interactions  riches  avec  la  visualisation 

•  Exploration  :  Faciliter  la  decouverte  d’ information  nouvelle 

•  Analyse  :  Encourager  les  utilisateurs  a  reunir  les  evidences  pour  en  former  une  « 
histoire  » 

La  Section  3  caracterise  la  communaute  de  la  recherche  globale  en  AV  en  presentant 
neuf  chercheurs  importants  et  les  institutions  auxquelles  ils  sont  affilies.  Ceci  inclut  une 
vue  d’ensemble  du  Consortium  en  Analytique  Visuelle  d’Amerique  du  Nord  et  des 
developpements  paralleles  en  Europe. 

La  Section  4  contient  la  revue  de  la  litterature.  La  premiere  sous-section  enumere  neuf 
joumaux  et  conferences  visant  la  communaute  de  la  recherche  en  AV,  soulignant  en 
particulier  fimportance  de  la  conference  Visual  Analytics  Science  and  Technology 
(VAST)  de  IEEE.  Puisque  beaucoup  d’articles  importants  sont  publies  independamment 
ou  rendus  disponibles  sous  forme  de  pretirages  sur  internet,  la  Section  4.1.4  enumere 
egalement  ces  instituts  de  recherche  qui  maintiennent  des  pages  avec  du  contenu 
important  en  AV.  Suite  a  des  consultations  independantes  avec  des  experts  en  matiere 
de  TSM,  la  section  4.2  identifie  les  possibility  suivantes  en  AV  qui  presentent  un 
interet  particulier : 
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•  Eveil  situationnel  geo-spatiale-temporelle 

•  Exploitation  de  donnees  geo-spatiales-temporelles 

•  Representation  de  1’ incertitude  geo-spatiale-temporelle 

•  Analyse  de  la  couverture  geo-temporelle 

•  Previsions  et  analyse  par  simulation 

•  Surveillance  a  l’aide  de  capteurs  et  de  reseaux 

•  Analyse  de  reseaux  sociaux 

•  Collaboration 

•  Exploration  de  textes  et  de  nouvelles  en  continu 

•  Interfaces  utilisateurs  et  utilisabilite  pour  l'analytique  visuelle 

•  Principes  de  conception  de  l’analytique  visuelle 

La  Section  4.3  identific  et  presente  brievement  environ  70  publications  qui  abordent  ces 
sujets. 

La  Section  5  passe  en  revue  quinze  produits  d’AV  qui  sont  deja  disponibles  et  qui 
possedent  certaines  ou  toutes  ces  fonctionnalites.  Pour  chacun  de  ces  produits,  est 
fourni  un  tableau  contenant  une  capture  d’ecran  ou  image,  une  critique  breve  de  ses 
possibility  et  limitations,  une  evaluation  de  sa  conformance  a  l’Analytique  Visuelle 
(basee  sur  les  quatre  criteres  ci-haut)  et  des  liens  vers  de  l’information  supplementaire. 
Lorsque  cela  s’avere  possible,  des  extraits  video  sont  ajoutes  car  ils  foumissent  une 
meilleure  idee  des  interactions  utilisateur  qui  sont  tres  importantes  pour  l’Analytique 
Visuelle. 

L'annexe  B  est  une  bibliographic  annotee  des  articles,  sites  internet,  et  produits 
mentionnes  dans  ce  document.  Chaque  entree  est  composee  de  la  reference,  du  lien 
internet  si  disponible  et  d’une  evaluation  de  la  pertinence  et  de  la  valeur  de  la  dite 
entree.  Lorsque  cela  etait  possible,  des  copies  numeriques  des  documents  et  videos 
mentionnes  ont  ete  foumies  avec  ce  rapport.  Ces  fichiers  sont  identifies  par  leur  noms 
dans  la  bibliographic.  La  base  de  donnees  EndNote  employee  pour  produire  la 
bibliographic  a  ete  egalement  foumie. 
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INTRODUCTION 


This  document  identities  and  characterizes  “Visual  Analytics”  technologies  and 
products  relevant  to  maritime  domain  awareness  in  the  Canadian  Forces  regional  joint 
operations  centres  (RJOCs).  It  is  one  of  two  deliverables  (the  second  deliverable  is 
[31])  from  the  Visual  Analytics  for  Maritime  Awareness  (VAMA)  contract. 

Sections  1 . 1  and  1 .2,  which  are  repeated  in  both  documents,  give  a  brief  overview  of  the 
context  and  scope  of  the  contract. 

Task  1  investigations,  described  in  Sections  2  through  5,  include: 

•  A  discussion  of  what  the  term  “Visual  Analytics”  means,  some  of  the  challenges 
associated  with  that  term,  and  a  taxonomy  of  visual  analytics  technologies  and 
techniques  (Section  2). 

•  An  overview  of  the  emerging  Visual  Analytics  research  community  (Section  3). 

•  A  guide  to  the  Annotated  Bibliography  and  overview  of  the  academic  papers  that 
were  found  (Section  4). 

•  A  review  of  the  Visual  Analytics  products  that  were  found  (Section  5). 

•  The  Annotated  Bibliography  itself  (Appendix  B). 


Motivation 

The  Canadian  Forces  (CF)  has  a  mandate  to  be  aware  of  all  that  is  happening  (i.e.  all 
“situational  facts”)  in  Canada’s  maritime  approaches.  It  is  based  on  the  need  to  protect 
Canada  from  attack,  defend  Canada’s  sovereignty,  detect  illegal  activities,  and  support 
search  and  rescue  activities.  The  mandate  became  more  challenging  in  the  past  decade, 
as  commercial  shipping  became  a  potential  threat. 
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Regional  Joint  Operations  Centres  (RJOCs)  in  Halifax  and  Esquimalt  maintain  24/7 
watch  over  Canada’s  three  oceans  in  support  of  the  mandate.  They  do  this  by  extracting 
situational  facts  from  a  variety  of  sensor  data  streams  and  analysis  tools  that  have  been 
documented  elsewhere  (e.g.  [33]).  The  main  challenges  facing  the  RJOCs  are  not  a  lack 
of  data  -  in  some  ways  the  RJOCs  are  drowning  in  data  -  but  rather: 

•  The  data  has  often  been  degraded  by  ambiguities  and  data-entry  errors,  requiring 
operators  to  spend  large  amounts  of  time  cleaning  up  the  data  to  reveal  the 
underlying  facts.  Efforts  are  currently  being  made  by  DRDC  Centre  for 
Operational  Research  and  Analysis  (CORA)  to  address  this  with  the  Attribute 
Correction  Engine  (ACE)  and  the  RMP  Analysis  Toolset  (RAT). 

•  The  data  has  too  often  been  delivered  to  the  operators  as  individual  dots  on 
screens,  rather  than  being  first  resolved  into  tracks.  This  was  recently  addressed 
by  the  Multi-Sensor  Integration  within  a  Common  Operating  Environment 
(MUSIC)  Technology  Demonstration  Program  led  by  DRDC  Atlantic. 

•  A  huge  fraction  of  the  information  presented  to  the  operators  is  mundane,  and  no 
attempt  is  made  to  draw  attention  to  the  non-mundane.  Automated  knowledge 
management  solutions  to  this  are  being  addressed  in  the  Automated  Reasoning  for 
Maritime  Anomaly  Detection  (ARMAD)  contract  at  DRDC  Valcartier.  The 
Maritime  Domain  Analysis  through  Collaboration  and  Interactive  Visualization 
(MDACIV)  project  at  DRDC  Valcartier  is  exploring  user-interactive  solutions. 

The  Visual  Analytics  for  Maritime  Awareness  (VAMA)  contract  is  searching  for  state- 
of-the-art  visual  analytics  approaches  and  technologies  to  support  the  visualization, 
interaction  with,  and  analysis  of  the  RMP.  The  study  includes  an  assessment  of  the 
technologies,  a  survey  of  the  scientific  literature,  and  an  analysis  of  user  requirements. 

Previous  DRDC  studies  have  documented  RJOC  activities  (e.g.  [29],  [33])  potential 
requirements  ([27];  [1 13]),  and  relevant  visualization  technologies  ([32];  [28]).  These 
studies  highlighted  challenges  such  as  the  following  for  both  automated  and  human 
detectors: 

•  Contact  reports  typically  have  low  cadence  (i.e.  are  infrequent); 

•  Some  information  sources  (the  cleanest  ones)  are  voluntary  and  thus  spoofable, 
though  there  are  few  scenarios  where  spoofing  would  be  advantageous  to  an 
attacker; 

•  Some  parts  of  Canada’s  area  of  responsibility  are  poorly  covered  by  sensors; 

•  Some  important  information  streams  are  not  available  to  the  RJOCs  due  to  legal 
firewalls. 


1.2  VAMA  Contract  Scope 

The  scope  of  the  VAMA  contract  is  summarized  in  Figure  1-1.  Task  1  (recorded  in  this 
document)  is  a  literature  study  and  product  review  of  state-of-the-art  visual  analytics 
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technology,  recorded  in  an  annotated  bibliography.  Task  2  is  an  analysis  of  how  visual 
analytics  could  help  the  RJOCs  achieve  their  operational  objectives. 


Literature  Review 

T askl :  State-of-the-Art  Study 

Product  Review 


VAMA 


DND  Document  Review 

Task  2:  Requirements  Analysis  Scenario  Analysis 

RJOC  Site  Visits 


Figure  1-1  VAMA  will  lay  the  groundwork  for  Visual  Analytics  prototype  development 

The  VAMA  contract  includes  a  literature  review,  product  review,  and  requirements  analysis. 
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DEFINING  VISUAL  ANALYTICS 


Researchers  agree  with  the  following  definition  of  Visual  Analytics  from  Illuminating 
the  Path,  and  quote  it  frequently: 

“Visual  Analytics  is  the  science  of  analytical  reasoning  facilitated  by  2  | 

interactive  visual  interfaces.”  [ref  [13 1],  pg  4] 

The  following  subsections  refine  this  definition  by  discussing  the  normal  objectives, 
typical  behaviour,  and  minimal  capabilities  of  interactive  visual  interfaces  that  are  truly 
“Visual  Analytics”  systems,  and  then  proposing  a  taxonomy  of  the  field. 


Objectives 

Thomas  summarizes  the  goal  of  Visual  Analytics  interfaces  as  follows: 

“The  goal  of  visual  analytics  is  to  facilitate  this  analytical  reasoning  process 
through  the  creation  of  software  that  maximizes  human  capacity  to  perceive, 
understand,  and  reason  about  complex  and  dynamic  data  and  situations.  It  must 
build  upon  an  understanding  of  the  reasoning  process,  as  well  as  an  understanding 
of  underlying  cognitive  and  perceptual  principles,  to  provide  mission-appropriate 
interactions  that  allow  analysts  to  have  a  true  discourse  with  their  information. 

The  goal  is  to  facilitate  high-quality  human  judgment  with  a  limited  investment  of 
the  analysts’  time.”  [ref  [131],  pg  6] 

Note  the  following  features  in  this  statement: 

•  The  software  reaches  out  to  the  users,  seeking  a  good  “impedance  match” 
between  the  data  space  and  the  human  mind. 

•  Interactions  are  understood  to  be  mission-specific. 

•  Interfaces  are  based  on  scientific  models  of  human  cognition. 
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2.2  Common  Visual  Analytic  Patterns 

Typical  activities  and  applications  of  Visual  Analytics  are  [refs  [61],  [130]]: 

•  Synthesizing  information  into  knowledge 

•  Deriving  insight  from  massive,  dynamic,  and  often  conflicting  data 

•  Detecting  the  expected 

•  Discovering  the  unexpected 

•  Providing  timely,  defensible,  and  understandable  assessments 

•  Communicating  assessments  effectively  for  action 

Schneidennan  [ref  [1 18]]  observed  that  interactive  visualization  tools  always  support 
the  following  sequence  of  activities: 

1 .  Overview  of  the  data  space 

2.  Zoom 

3.  Filter 

4.  Details  on  demand  (drill-down) 

5.  Relate  (view  links) 

6.  Show  history 

7.  Extract  information 

Keim  [77]  suggests  that  the  Visual  Analytics  has  the  following  slightly  different 
sequence: 

1.  Analyse  First 

2.  Show  what  is  Important 

3.  Zoom 

4.  Filter 

5.  Analyse  Further 

6.  Details  on  Demand 

Riveiro  [107]  finds  the  “foraging  loop”  [131]  a  more  appropriate  pattern  for  describing 
visual  analytic  sequencing.  Her  model  (see  Figure  2-1)  suggests  the  following 
sequence: 

1.  Frame  the  data 

2.  Question  the  frame 

3.  a)  Elaborate  (add  detail  to)  the  frame  or 
b)  Discard  and  re-frame 


2-2 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


MDA 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


Elaboration  Refranung 

cycle  cycle 


Preserve 

Figure  2-1  Visual  Analytics  as  a  Framing  and  Elaboration  Cycle. 

Riveiro  proposes  a  dual-cycle  visual  analytics  cycle,  focused  on  framing  the  data.  If  the  frame  is  working,  an 
elaboration  cycle  is  used  to  clean  up  what  is  in  the  frame  and  insert  more  insights.  If  the  frame  is  not 
working,  a  new  frame  is  sought  so  the  data  can  be  reframed. 

The  published  videos  of  Jigsaw  [125],  Starlight  [101],  and  In-Spire  [100]  are 
characterized  by  the  following  sequence: 

1 .  Query  or  visualize  the  evidence  (in  multiple  visual  domains) 

2.  Find  patterns  and  filter  the  evidence 

3.  Visualize  what  links  to  the  evidence 

4.  Find  patterns  and  filter  the  links 

5.  Follow  the  links  to  new  evidence 

6.  Repeat (1-5) 

Narrowing  the  Definition 


f  Elaborate  a  framed 
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Judge  plausibility 
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Compare  * 
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Seek  a  new  _ 
frame  's— ' 


‘Ca 


Because  Visual  Analytics  is  enjoying  a  surge  of  attention  from  researchers,  there  is  a 
risk  that  the  meaning  of  the  term  will  be  diluted  through  over-use.  A  visualization 
paper  by  Bak  for  example  [9]  is  falsely  labeled  as  visual  analytics.  This  section  thus 
proposes  ways  to  tighten  up  definition  (2-1). 

Jim  Thomas  [130]  narrows  the  definition  of  Visual  Analytics  by  noting  that  the 
following  system  features  that  are  not  (by  themselves)  indicative  of  Visual  Analytics: 

•  Large  graph  structures  with  no  labels 

•  Heat  maps  with  no  labels 

•  Search  and  retrieval  systems 
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•  Charts  with  no  interaction 

•  Images  with  no  semantic  interpretation 

•  Stand  alone  images  that  do  not  tell  a  story 

It  is  thus  not  enough  to  have  a  visualization  strategy;  the  tool  must  provide  meaningful 
interaction  with  the  visualization,  that  leads  to  a  labeled  view  that  tells  a  story.  And  the 
tool  must  facilitate  the  discovery  of  the  story  -  thus  Photoshop,  though  capable  of 
presenting  a  complex  visual  story,  is  not  a  Visual  Analytics  tool. 


We  thus  amplify  the  definition  (2-1)  of  Visual  Analytics  tools  by  requiring  that  each 
must  be: 


•  A  Visualization  tool:  the  tool  must  depict  complex,  abstract,  or 
voluminous  information  using  substantially  sophisticated  or  abstract 
views. 

•  An  Interactive  tool:  users  must  be  able  to  modify  the  visualization, 
filter  data,  assess  hypotheses,  infer  meaning,  and  test  the  truth. 

•  An  Exploratory  tool:  the  tool  must  facilitate  the  discovery  of 
unexpected  or  hidden  meanings. 

•  An  Analytical  tool:  users  must  be  able  to  assemble  evidence,  record 
ideas,  and  build  an  underlying  “story”  that  gives  meaning  to  the  data. 

•  A  Scientific  tool:  the  tool  must  be  based  on  scientific  cognitive  and 
perceptual  models,  and  amenable  to  scientific  investigation. 


(2-2) 

(2-3) 

(2-4) 

(2-5) 

(2-6) 


2.3.1  Alternative  Terminology 

Caution  must  be  used  with  the  tenn  “Visual  Analytics”  because  it  is  copyrighted  [ref 
[137]].  Thus  the  National  Visualization  and  Analytics  Center  (NVAC)  uses  the  phrase 
“visualization  and  analytics”  rather  than  “visual  analytics.”  The  tenn  “Visual 
Analytics”  continues  to  be  widely  used,  however,  perhaps  because  of  Illuminating  the 
Path  [131]. 

2.4  Visual  Analytics  Taxonomies  and  Typologies 

Experts  in  the  science  of  classification  [8]  distinguish  between  the  “taxonomies”  and 
“typologies”  as  follows: 

•  Typology:  a  classification  procedure  in  which  only  the  concepts  are  classified. 
Thus  each  item  has  multiple  characteristics  that  define  its  classification. 

•  Taxonomy:  a  classification  procedure  in  which  only  empirical  entities  are 
classified.  Thus  each  item  belongs  to  exactly  one  class. 
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There  is  no  reason  to  believe  that  a  true  taxonomy  of  Visual  Analytics  is  possible  or 
even  desirable,  because  the  topic  is  fundamentally  aimed  at  combining  a  variety  of 
views,  bringing  together  many  different  types  of  data,  and  addressing  multiple 
operational  goals. 

Some  example  typologies  from  Jim  Thomas  (Figure  2-2  and  Figure  2-3)  and  Georgia 
Tech  (Figure  2-4)  are  shown  below.  The  wording  in  Figure  2-2  has  been  changed  as 
follows: 

•  Descriptions  of  “Goals”  have  all  been  changed  to  verb  fonn 

•  “Defraud”  and  “Deceive”  have  been  combined 

•  “Collaborate”  has  been  inserted 

•  The  “Uncertainty  /  Imprecision”  data  type  has  been  inserted 

The  Starlight  tool,  one  of  the  first  Visual  Analytics  applications  to  be  developed,  has  at 
its  core  a  useful  taxonomy  of  information  shapes,  as  shown  in  Figure  2-5.  By  making 
the  taxonomy  small,  while  still  supporting  all  types  of  relationship,  the  designers 
empower  Starlight  to  display  almost  any  type  of  information  relationship  using  only  six 
types  of  display  axis. 
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Figure  2-2  Jim  Thomas’s  Typology  of  Visual  Analytics,  Goals,  Domains,  and  Data  Types 

Jim  Thomas,  a  central  figure  in  Visual  Analytics  development,  proposed  this  typology  (he  called  it  a  “taxonomy”) 
based  on  four  characteristics  that  each  application  has:  data  types,  analysis  goals,  application  domain,  and  technology 
employed  [based  on  [129]].  The  “technology”  branch  is  expanded  in  Figure  2-3. 
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Figure  2-3  Jim  Thomas’s  Typology  of  Visual  Analytics  Technologies 

This  completes  the  typology  introduced  in  Figure  1-1.  [based  on  [129]]. 
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Figure  2-4  Georgia  Tech  Typology  of  Visual  Analytics  Technologies 

Georgia  Tech  uses  this  typology  to  organize  their  teaching  material,  [based  on  [48]] 
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Figure  2-5  Starlight  Taxonomy  of  Information  Relationships 

Starlight  achieves  very  flexible  information  display  by  reducing  all  types  of  information  relationships  to  these  six 
classes.  Types  of  relationship  not  shown  here  (e.g.  similarity  of  meaning  in  text)  can  be  displayed  if  they  can  be 
reduced  to  one  of  these  classes  (e.g.  a  vector  -space  that  is  representative  of  textual  meaning),  (from  Figure  2-5  of 
[28]). 
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VISUAL  ANALYTICS  RESEARCH  COMMUNITY 


The  following  sections  describe  key  agencies  and  departments  that  are  coordinating  and 
implementing  Visual  Analytics  research,  including: 

•  Individuals  who  are  influential  in  the  field,  or  strategic  to  the  interests  of  DRDC 
(Section  3.1) 

•  Coordinating  agencies  and  collaborative  structures  (Section  3.2) 


Key  People 

This  section  gives  a  brief  introduction  to  some  of  the  most  influential  individuals  in  the 
Visual  Analytics  (VA)  research  community. 
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3.1.1  Dr.  Jim  Thomas 


Affiliation: 

•  Laboratory  Fellow  at  Pacific  Northwest  National 
Laboratory  (PNNL). 

Notable  For: 

•  Founding  Director,  National  Visual  and  Analytics 
Center  (NVAC)  (resigned  August  2009). 

•  Co-wrote  “Illuminating  the  Path”  [131]  thus 
setting  the  agenda  for  visual  analytics  research. 

•  Set  up  the  network  of  centers  of  excellence. 

•  Knows  everyone  in  the  VA  community. 


•  PNNL  is  notable  for  Starlight  and  InSpire. 


3.1.2  Dr.  Joe  Kielman 


Affiliation: 

•  Department  of  Homeland  Security  (DHS). 

•  Director,  Research  Futures. 

•  Science  advisor  in  the  office  of  the  Under 
Secretary  for  Science  and  Technology. 

Notable  For: 

•  Single  most  important  US  customer  representative. 

•  He  sees  the  objective  of  Visual  Analytics  as  ‘to 
seek  ways  to  deliver  “designer  information” — up- 
to-date,  customized  or  tailored  knowledge 
delivered  just  as  needed.’  [78] 
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3.1.3  Prof.  David  Ebert 


Affiliation: 

•  Director,  Purdue  University  Regional 
Visualization  and  Analytics  Center  (PURVAC). 

•  PURPL  :  Purdue  University  Rendering  and 
Perceptualization  Lab. 


Research  Focus: 

•  Novel  visualization  techniques,  visual  analytics, 
volume  rendering,  information  visualization, 
perceptually-based  visualization,  illustrative 
visualization,  and  procedural  abstraction  of 
complex,  massive  data. 

Notable  For: 

•  Purdue  is  the  senior  and  managing  lab  in  Visual 
Analytics  for  Command,  Control,  and 
INteroperability  Environments  (VACCINE). 


m 


3.1.4  Prof.  Brian  Fisher 


Affiliation: 


•  Associate  Professor,  Simon  Fraser  University 
(SFU)  and  University  of  British  Columbia  (UBC). 

•  Director,  SFU  Spatial  Cognition  and  Interactive 
Expertise  in  Natural  and  Computational 
Environments  (SCIENCE  Lab). 

•  Associate  Director,  SFU  Visual  Analytics 
Research  Lab  (VAR  Lab). 

Research  Focus: 

•  Cognitive  and  social  science-based  approaches  to 
development  and  evaluation  of  analytics  systems 
and  processes. 


Notable  For: 

•  The  leading  Canadian  academic  in  visual  analytics. 
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3.1.5  Prof.  John  Stasco 


Affiliation: 

• 

Director  of  the  Georgia  Tech  component  of  the 
Southeastern  Regional  Visualization  and  Analytics 
Center  (SRVAC). 

• 

Director  of  the  Information  Interfaces  Research 
Group. 

Research  Focus: 

• 

central  focus  of  a  number  of  our  projects  is  the 
creation  of  information  visualization  and  visual 
analytics  tools  to  help  people  understand  and 
analyze  large  data  sets. 

Notable 

For: 

• 

Georgia  Tech  is  notable  for  Jigsaw  and  the  Global 
Terrorism  Database. 

3.1.6  Prof.  Haesun  Park 


Affiliation: 

•  Director  of  Foundations  of  Data  and  Visual 
Analytics  (FODAVA). 

•  Professor  and  Associate  Chair,  Georgia  Tech. 

Research  Focus: 

•  Numerical  Algorithms,  Data  and  Visual  Analytics, 
Bioinformatics,  Text  Analysis. 

Notable  For: 

•  Has  not  published  any  Visual  Analytics  papers. 
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3.1.7  Dr.  Richard  May 


Affiliation: 


•  Director,  National  Visual  and  Analytics  Center 
(NVAC)  (as  of  August  2009). 

•  Previous  co-chair  IEEE  Visual  Analytics  Science 
and  Technology  (VAST)  conference. 


Research  Focus: 

•  Designing  new  technologies  and  protocols  to 
generate  and  interact  with  complex,  massive  data 
sets. 

•  He  has  conducted  research  in  video  and  image 
processing,  information  visualization,  virtual  and 
mixed  reality  and  visual  analytics. 


3.1.8  Drs.  Gennady  and  Natalia  Andrienko 


Affiliation: 


(both)  Research  fellows,  Fraunhofer  IAIS  , 
Germany. 

(G)  Chair  of  the  Commission  on  Geo  Visualization 
of  the  International  Cartographic  Association: 


Research  Focus: 

•  Visualisation  of  spatial  and  spatio-temporal  data  as 
a  tool  for  visual  analytics,  exploratory  data 
analysis,  problem  solving,  and  decision  making. 

•  Co-editors  of  the  Geovisualization  of  Dynamics, 
Movement  and  Change  special  issue  of 
Information  Visualization . 

•  Co-authors  of  Exploratory  Analysis  of  Spatial  and 
Temporal  Data,  (Springer- Verlag,  2005). 
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3.2  Coordinating  Agencies  and  Collaborations 

The  following  key  agencies  and  coordinating  bodies  in  Visual  Analytics  are  described 
in  this  subsection: 

•  US  Department  of  Homeland  Security  (DHS)  Section  3.2.1. 

•  National  Visual  and  Analytics  Centre  (NVAC)  Section  3.2.2. 

•  The  Visual  Analytics  Consortium  (VAC)  Section  3.2.3. 

•  Visual  Analytics  for  Command,  Control,  and  INteroperability  Environments 
(VACCINE)  and  Foundations  on  Data  Analysis  and  Visual  Analytics  (FODAVA) 
Section  3.2.4. 

•  Visual  Analytics  organizations  in  Europe:  Section  3.2.5. 

3.2.1  US  Department  of  Homeland  Security 

The  Research  and  Development  agenda  for  Visual  Analytics  was,  from  the  beginning, 
motivated  by  the  following  seven  strategic  goals  and  objectives  of  the  US  Department 
of  Homeland  Defense  (DHS):  awareness,  prevention,  protection,  response,  recovery, 
service  (to  normal  activities),  and  organizational  excellence  (within  the  DHS)  (page  3 
of  [13 1]).  Five  years  later,  the  DHS  remains  the  principal  funding  source  for  the  various 
centers  of  excellence. 

Fisher  (pg  2  of  [41])  lists  the  following  DHS  presidential  directives  (paraphrased  here 
based  on  [56])  as  being  of  direct  relevance  to  visual  analytics: 

3:  Create  a  Homeland  Security  Advisory  System  to  evaluate  and  report  on  the 
current  risk  of  terrorist  acts. 

5:  Provide  a  single,  comprehensive  national  incident  management  system  for  the 
prevention,  preparation,  response,  and  recovery  from  terrorist  attacks,  major 
disasters,  and  other  emergencies. 

7:  Identify  and  prioritize  critical  infrastructure  and  protect  them  from  terrorist 
attacks. 

8:  Establish  standard  and  comprehensive  approach  to  national  planning  so  as  to  be 
prepared  to  prevent  and  respond  to  threatened  or  actual  domestic  terrorist  attacks, 
major  disasters,  and  other  emergencies.  . 

9:  Defend  the  agriculture  and  food  system  against  terrorist  attacks,  major  disasters, 
and  other  emergencies. 

10:  Provide  support  for  threat  awareness,  prevention,  protection,  surveillance, 
detection,  response  and  recovery  against  the  biological  weapons  threat. 
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2 1 :  Be  prepared  to  respond  to  the  public  health  and  medical  needs  of  the  American 
people  in  the  case  of  a  catastrophic  health  event  through  continual  and  timely 
flow  of  information. 

23:  The  National  Cyber  Security  Initiative,  which  presumably  (this  topic  is  not 
addressed  on  the  website)  searches  for  vulnerabilities  in  the  American  computer 
network  infrastructure,  detects  incursions,  and  supports  network  defense  activities 
in  response  to  an  attack. 

To  achieve  this  mandate,  the  DHS  budget  has  two  line-items  specifically  for  Visual 
Analytics  through  VACCINE  and  FODAVA  (Section  3.2.4): 

•  Approximately  $4.5M  per  year  for  basic  future  research  (shared  with  physics- 
based  simulations  as  shown  in  Figure  3-1). 

•  Approximately  $3M  per  year  funding  to  university  Centers  of  Excellence  for 
Command,  Control,  and  Interoperability. 

Other  DHS  funding  mechanisms,  such  as  “International  and  Interagency  Programs”  (pg 
161  of  [35])  are  also  available.  As  a  major  funding  agency,  DHS  has  a  strong  influence 
on  how  the  visual  analytics  community  is  organized,  and  on  what  topics  they  do 
research.  Thus  for  example  the  re-organizations  listed  in  Sections  3.2.3  and  3.2.4  were 
influenced  by  DHS  expectations  and  preferences. 

As  discussed  in  Section  3.2.4,  the  National  Science  Foundation  now  also  provides 
funding  for  visual  analytics. 


PPA 

Thrust  Area 

Program 

FY  08 
(RE) 
($000) 

FY09 

(PB) 

($000) 

FY10 

(Plan) 

($000) 

FY11 

(Plan) 

($000) 

FY12 

(Plan) 

($000) 

FY13 

(Plan) 

($000) 

Control  & 
Interoperability 

Interoperability 
and  Compatibility 

OIC 

10,290 

11,084 

8,270 

6,477 

4,671 

7,144 

Cyber  Secunty 

Cyber  Security  Research  Tools 
and  Techniques 

7,160 

6,971 

6,000 

6,400 

7,000 

7,200 

Information  Infrastructure  Security 

9,880 

11,181 

8,577 

8,573 

6,152 

7,012 

Next  Generation  Technologies 

2,841 

0 

700 

800 

2,975 

3,134 

Basic  Future 
Research 

Visual  Analytics  and  Physics- 
based  Simulation 

4,235 

4,536 

4,744 

4,813 

5,028 

5,142 

Figure  3-1  Homeland  Security  Budget  Line  Item  for  University  Visual  Analytic  Research 

Basic  future  research  in  visual  analytics  is  funded  under  the  Command,  Control,  &  Interoperability  (CCI)  segment  of 
the  DHS  Science  and  Technology  budget.  Shown  here  is  an  excerpt  from  [35]. 
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3.2.2  National  Visualization  and  Analytics  Center 

The  National  Visualization  and  Analytics  Center  (NVAC)  was  established  in  2004  by 
the  Department  of  Homeland  Security  to  develop  the  tools  to  identify  and  prevent 
terrorist  attacks.  NVAC  is  part  of  Pacific  Northwest  National  Labs  (PNNL)  and  is 
located  in  Richland  Washington.  The  NVAC  assembled  a  multidisciplinary  panel 
representing  government,  academia  and  industry  to  develop  a  Research  and 
Development  Agenda  [131]  to  define  the  directions  and  priorities  for  future  visual 
analytics  developments.  NVAC  inherited  two  visual  analytic  tools,  Starlight  and  In- 
Spire,  from  Pacific  PNNL  and  immediately  started  developing  more. 

The  NVAC  [92]  lists  the  following  research  areas  that  they  are  currently  pursuing: 

•  Large  Scale  Graph  Perception 

•  Analytic  Discourse  and  Scenario  Understanding 

•  Mobile  Visualization  Platforms 

•  Statistical  and  semantic  information  representations  and  transformations 

•  High  dimensional  infonnation  synthesis 

•  Visualization  and  Interaction  paradigms 

•  Temporal  and  Spatial  visual  analytics 

•  Information  packaging  and  dissemination 

•  Calibration  and  Validation  of  Information 

•  Collaboration  technologies  for  distributed  visual  analytics 

•  Architectures  for  Information-Processing  and  visual  analytics 

3.2.3  VAC  Consortium 

Once  the  NVAC  was  established,  it  immediately  started  encouraging  the  establishment 
of  smaller  “regional”  Visual  Analytics  Centers  (VACs)  and  Centers  of  Excellence 
(COEs).  Principal  members  of  the  resulting  consortium  of  academic  centers  are  shown 
in  Figure  3-2. 

Under  the  leadership  of  the  NVAC,  a  consortium  of  VA  capability  suppliers  (including 
industry  members)  and  VA  capability  users  (both  government  and  industry)  was 
formed.  The  group  meets  at  least  twice  a  year  to  “promote  a  high-level  of  information 
exchange  between  NVAC/RVAC  science  and  technologies  and  all  suppliers  of 
information  and  visual  analytics  and  technology  solutions”  [96]. 
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VAC  Consortium 


MIAS  COE  Urbana 


VAC  Name 

Example  Person 

Jim  Thomas 

Example 

Achievement 

NVAC  PNNL  Washington 

ex- Starlight 

NVAC 

ln_SPIRE 

NEVAC  Penn  State 

Alan  MacEachren 

ConceptVist 

PURVAC  Purdue 

David  Ebert 

Emergency  Response 

John  Stasko 

Global  Terrorism  Database 

SRVACUNC,  Georgia  Tech 

Jigsaw 

h 

1  VAC  . 

Visual  Analytics  Digital  Library 

[  .  CNVAC  SFU  UBC 

Brian  Fisher 

CZSaw 

PARVAC  U  Washington 

Mark  Haselkorn 

Stanford 

Pat  Hanrahan 

CKID  U  S  Cal 

KOJAK 

DyDAN  COE  Rutgers 

Laboratory  for  Port  Security 

1  COE 

CERATOPS  U  Pittsburg 

Industry 

\  .  Boeing 

Future  Point 
Microsoft 
\'\  Quantum  4D 
Tableau 

Figure  3-2  Research  Centers  in  the  Visual  Analytics  Consortium 

The  VAC  consortium  includes  a  network  of  Visual  Analytics  Centers  (VACs)  coordinated  by  the  National  Visual  and 
Analytics  Center  (NVAC)  in  Richland  Washington,  (based  on  [41])  This  diagram  does  not  show  the  VA  capability 
users  such  as  DHS. 


3.2.4  VACCINE  and  FODAVA 

The  DHS  re-organized  the  funding  mechanisms  for  Visual  Analytics  in  2008  and  2009, 
focusing  on  the  visual  analytics  collaborations  illustrated  in  Figure  3-3: 

•  Most  of  the  RVACs  are  being  transitioned  into  the  Visual  Analytics  for 
Command  Control  Interoperability  Environments  (VACCINE)  [41]  which  is 
funded  by  DHS. 
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Georgia  Tech 
Cornell 


Figure  3-3  VACCINE,  NVAC,  IDS,  and  FODAVA  in  2009 
VACCINE  is  the  DHS-funded  umbrella  organization  for  12  partners  focused  on  Command,  Control,  and 
Interoperability  requirements  of  DHS.  FODAVA  has  9  partners  focused  on  more  foundational  visual  analytic  science, 
funded  by  both  DHS  and  NSF  (based  on  [41]).  The  Institute  for  Discrete  Sciences  (IDS)  is  a  group  of  four  centers  of 
excellence  co-funded  by  DHS. 

•  The  SRVAC  and  some  university  partners  are  now  under  Foundations  on  Data 
Analysis  and  Visual  Analytics  (FODAVA)  [43]  which  is  funded  by  both  DHS 
and  the  US  National  Science  Foundation  (NSF). 

•  Some  COEs  are  now  part  of  the  Institute  for  Discrete  Sciences  (IDS)  which  is 
funded  by  DHS. 

VACCINE,  led  by  Purdue,  is  focused  on  seven  Command  Control  Interoperability 
(CCI)  directives,  and  has  two  types  of  program: 

•  4  long-term  research  streams  (“pillars”). 

•  15  shorter-term  “Mission  Guided  Projects”. 

FODAVA,  led  by  Georgia  Tech,  is  more  focused  on  scientific  foundations  of  the  data 
and  visual  analytics  fields  and  advancing  the  state-of-the-art. 
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The  NVAC  and  the  four  agencies  of  the  Institute  for  Discrete  Sciences  (IDS)  continue 
to  be  funded  by  DHS. 


.5  Visual  Analytics  in  Europe 

A  2007  paper  by  Kohlhammer  [79]  indicates  that  at  that  time  there  were  no  significant 
Visual  Analytics  initiatives  in  Europe.  In  October  2008,  a  web  page  at  the  University 
of  Konstanz,  Germany  [132]  announced  the  initiation  of  “VisMaster,”  a  European 
initiative  in  Visual  Analytics  involving  17  partner  agencies  in  9  European  countries. 
This  is  repeated  in  an  IEEE  article  in  2009  [80].  The  “News”  page  of  the  Konstanz  site, 
however,  has  not  been  updated  since  March  2009,  and  the  links  that  it  offers  often  make 
no  reference  to  VisMaster  or  Visual  Analytics.  Clearly  VisMaster  is  still  ramping  up. 

The  following  VisMaster  sites  are  worthy  of  note,  though  not  all  describe  their  research 
as  “visual  analytics”: 

•  Fraunhofer  Institute  for  Computer  Graphics  Research  (IGD)  in  Gennany  [65]:  A 
core  research  topic  is  Visual  Analytics  for  interactive  visualization  and  intelligent 
complexity  reduction,  which  they  understand  to  be  an  interdisciplinary 
combination  of  data  mining  and  infonnation  visualization.  Notable  researcher: 
Jorn  Kohlhammer. 

•  Fraunhofer  Institute  for  Intelligent  Analysis  and  Infonnation  Systems  (IAIS)  in 
Germany  [59]:  Focused  on  temporal  and  geospatial  data,  including  people,  cars, 
animals,  eye  movements,  weather  records,  earthquakes,  or  traffic  accidents. 
Notable  researcher:  Gennady  Andrienko. 

•  Chair  for  Databases,  Data  Analysis  and  Visualization,  University  of  Konstanz  in 
Germany  [133]:  Focused  on  techniques  for  interactive  mass  data  analysis  for  very 
large  multidimensional  and  geographic  data  sets,  clustering  methods  and  index 
structures  for  high-dimensional  spaces,  and  multimedia  similarity  search  (2D  and 
3D).  Researcher  leader:  Daniel  A.  Keim. 

•  AVIZ,  Institut  National  de  Recherche  en  Informatique  et  en  Automatique 
(INRIA),  in  France  [66]:  A  multidisciplinary  team  trying  to  improve  the  analysis 
and  visualization  of  large  and  complex  datasets  by  combining  analysis  methods 
with  interactive  visualizations.  The  focus  is  on  the  visual  analysis  of  large 
networks  (on  the  order  of  one  million  vertices  and  millions  of  edges)  and  time 
series  (logs  with  billion  of  records  captured  continuously  in  real-time).  Research 
leader:  Jean-Daniel  Fekete. 
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4  LITERATURE  REVIEW 


The  following  subsections  review  the  literature  available  on  visual  analytics  that  is 
relevant  to  maritime  domain  analysis.  Section  4. 1  begins  with  a  review  of  key  sources, 
such  as  journals,  conference  proceedings,  and  websites  of  research  centers.  Information 
sources  associated  with  commercial  companies  are  left  to  Section  5.  Section  4.3 
provides  a  topical  review  of  the  literature.  Section  4.3.8  provides  a  brief  introduction  to 
the  Annotated  Bibliography  which  is  in  Appendix  B. 

4.1  Sources  for  Research  Papers 

The  literature  survey  was  conducted  mostly  via  internet  links  indexed  by  Google  and 
Google  scholar.  Visual  Analytics  journals  (Section  4.1.1)  and  conferences 
(Section  4. 1 .2)  provided  rich  lists  of  links,  and  each  paper  provided  further  links  in  the 
citations.  The  websites  of  key  research  institutes  (Section  4. 1 .3)  were  also  helpful. 


4.1.1  Visual  Analytics  Journals 

Academics  generally  publish  Visual  Analytics  research  papers  in  one  of  the  following 
journals: 

•  IEEE  Transactions  on  Visualization  and  Computer  Graphics  (TVCG)  [64]  or 

•  Information  Visualization,  [102], 

The  NVAC  publishes  VAC  Views  magazine  [93]  which  provides  good  insights  into 
activities  around  the  VAC  Consortium. 

A  new  journal  specifically  dedicated  to  visual  analytics  has  been  announced,  but  the 
first  issue  will  not  be  published  until  2010: 

•  International  Journal  of  Visual  Analytics  for  Advanced  Infonnation  Management 
(IJVAAIM)  [50], 
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A  careful  review  revealed  no  Visual  Analytic  stream  in  any  of  the  Association  for 
Computing  Machinery  (ACM)  journals  (TAP,  TOCHI,  TKDD),  except  for  one  special 
issue  of  SIGKDD  Explorations  [1]. 


4.1.2  Visual  Analytics  Conferences 

The  following  conferences  and  workshops  are  focused  on  visual  analytics: 

•  IEEE’s  Visual  Analytics  Science  and  Technology  (VAST)  [62]. 

•  ACM’s  Workshop  on  Visual  Analytics  and  Knowledge  Discovery  (VAKD)  [7], 

Other  conferences  that  include  some  visual  analytics  papers,  or  seem  to  overlap  in  an 
important  way  are: 

•  IEEE’s  International  Conference  on  Coordinated  &  Multiple  Views  in 
Exploratory  Visualization  (CMV)  [60]. 

•  International  Cartographic  Association’s  Geospatial  Visual  Analytics  Workshop 
[67]  September  2008. 

•  SPIE’s  Defense,  Security,  &  Sensing  Conference  has  a  small  Visual  Analytics  for 
Homeland  Defense  and  Security  conference  track  [121], 

•  GI  Science’s  2006  workshop  on  Visualization,  Analytics  &  Spatial  Decision 
Support  [117]. 

•  A  one-day,  10-paper  workshop  within  EuroVis  2009  [63],  sponsored  by 
VisMaster. 

DHS  Science  and  Technology  branch  sponsors  the  following  conferences  to  discuss 
visual  analytics  research  activities  and  related  topics.  They  do  not  publish  proceedings: 

•  VAC  Consortium  meetings,  held  at  least  twice  a  year.  The  NVAC  Consortium 
webpage  [96]  provides  links  to  meeting  agendas  and  speaker  biographies,  but  no 
viewgraphs  or  conference  papers. 

•  DHS  University  Network  Summit,  held  annually.  A  glossy  brochure  [34]  is 
published  that  gives  agendas  and  very  brief  synopses  of  each  talk,  but  no 
viewgraphs  or  conference  papers. 

4.1.3  Teaching  Library 

The  Visual  Analytics  Digital  Library  [48]  provides  a  collection  of  about  240  lecture 
notes  and  homework  assignments  from  the  Georgia  Tech  center  of  excellence.  They  are 
organized  according  to  the  taxonomy  shown  in  Figure  2-4. 
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.4  Websites  of  Research  Institutes 

A  large  fraction  of  visual  analytics  research  is  done  through  VAC  consortium  labs,  as 
listed  in  Figure  3-3  and  in  [37].  Some  labs  have  web  pages  with  links  to  publications, 
as  summarized  here: 

•  National  Visual  and  Analytics  Center  (NVAC)  [95]  large-scale  graph 
perception,  analytic  discourse  and  scenario  understanding,  mobile  visualization 
platforms,  statistical  and  semantic  infonnation  representations  and 
transformations,  high  dimensional  information  synthesis,  visualization  and 
interaction  paradigms,  temporal  and  spatial  visual  analytics,  information 
packaging  and  dissemination,  calibration  and  validation  of  information, 
collaboration  technologies  for  distributed  visual  analytics,  architectures  for 
information-processing  and  visual  analytics 

•  Georgia  Tech  and  U  North  Carolina  (SRVAC)  [123]:  evidence  gathering, 
investigative  analysis,  hypothesis  building,  analytical  reasoning,  global  terrorism 
data  visualization,  multimedia  analysis,  image  and  video  visual  analysis, 
semantic  image  browser,  automatic  news  video  analysis,  Jigsaw  visualization  for 
investigative  analysis,  financial  analysis,  search  spaces  and  search  results,  Visual 
Analytics  Digital  Library  [48]. 

•  U.  Illinois  at  Urbana  (MIAS)  [91]:  meaning-based  information  retrieval, 
automated  text  analysis  using  contextual  patterns  and  probabilistic  topic  models, 
exploring  the  deep  web  (structured  infonnation  sources),  identifying  hidden 
themes  in  massive  datasets,  learning  the  meaning  of  pictures,  monitoring  online 
communities. 

•  Purdue  University  (PURVAC)  [104]  data  integration,  analysis  of  anonymized 
data,  disaster  response,  mobile  analytics  for  emergency  responders,  personnel 
tracking  and  video  capture  for  emergency  response,  sensor-driven  video 
analytics,  analysis  of  social  networks,  zoonotic  disease  spread,  time-space 
analysis  for  disease  surveillance,  and  network  flow  monitoring  and  forensics. 

•  Penn  State  (NEVAC)  [94]  knowledge-enabled,  geographically  aware  visual 
analytics,  FactXtractor  and  TexPlorer  for  infonnation  extraction  and  visualization 
in  text  documents,  FEMARepViz  and  Geo  Viz  Toolkit  for  automatic  information 
extraction  in  geo-temporal-attribute  visualization,  Health  GeoJunction  tracks 
indicators  of  infectious  disease  threats,  “Improvise”  provides  rapid  VA 
prototyping,  CiteSpace  tracks  trends  in  a  volatile  information  environment, 
SemanticNetSA  assesses  evidentiary  value  ConceptVista  provides  a  visual 
environment  for  ontology  development  NeoCITIES  Geo-Tools  provides  a  scaled- 
world  simulation  to  mimic  real-life  civic  emergencies. 

•  University  of  Washington  (PARVAC)[134]  distributed  cognition  and  VA  for 
Pacific  Rim  public  safety  and  security,  RimSimis  a  role-playing  collaboration 
testbed  for  simulated  emergency  scenarios,  a  geo-spatial  model  of  an  urban 
environment,  and  geo-spatial  pre-deployment  of  emergency  supplies. 
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•  SFU  /  UBC  (CNVAC)  [26]  Spatial  cognition  in  VA  environments:  seeing 
patterns  in  data,  enactive  cognition  in  VA  environments:  human-infonnation 
dialog  (sensemaking),  distributed  cognition  in  VA  environments:  collaborative 
and  competitive  analytics. 

•  Stanford  [122]  applications  drawn  from  transactional  analytics,  scalable  visual 
analysis  over  large  temporal  data  sets,  network  security  investigations, 
heterogeneous  information  spaces,  perceptual  efficiency  in  visual  attention  and 
cognitive  load. 

•  U.  Pittsburg  (CERATOPS)  [22]  infonnation  extraction  from  unstructured  text, 
detecting  opinion  and  sentiment  types,  automating  fine-grained  interpretation  of 
opinions,  exploiting  semantics. 

•  Rutgers  (DyDAn)  [36]  analysis  of  large,  dynamic  virtual  communities,  optimal 
decision-making  under  uncertainty,  Laboratory  for  Port  Security  developing  data 
analytic  tools,  inspection  strategies,  and  event  mitigation  for  Port  Security,  sensor 
management  for  nuclear  threat  detection,  cross  validation  of  data  in  a  privacy¬ 
preserving  manner,  information  theoretic  measures  for  biosurveillance, 
augmenting  infonnation  in  ontologies. 

•  U.  Southern  California  (CKID)  [57]  extract  semantic  graphs  from  large  data 
collections,  fuse  and  integrate  data  at  the  semantic  feature  level,  automated 
discovery  of  complex  relationships  in  semantic  graphs,  unstructured  text  analysis, 
uncertainty  quantification,  through  knowledge-based  inference  and  information 
theory. 


4.2  Domain  of  Interest 

This  section  proposes  a  set  of  visual  analytics  topics  that  are  of  particular  interest  for 
maritime  domain  awareness.  The  analysis  is  based  on: 

•  The  visual  analytics  typology  proposed  by  Jim  Thomas  and  presented  (with  some 
changes)  in  Figure  2-2  and  Figure  2-3. 

•  An  analysis  [30]  of  Canadian  Forces  requirements,  including  a  list  of  9  potential 
visual  analytics  applications. 

Table  4-1  shows  how  the  goals  and  data  types  can  be  associated  with  the  potential 
applications.  Note  the  following  patterns  in  the  table: 

•  Surveillance  Coverage  is  the  simplest  column  -  it  requires  the  fewest  data  types 
and  has  a  narrow  range  of  goals.  This  might  indicate  that  it  is  a  good  candidate 
for  early  implementation,  or  it  might  suggest  that  this  problem  is  too  simple  to  be 
called  Visual  Analytics. 

•  There  is  little  or  no  requirement  for  scientific  analysis,  audio  signal  analysis, 
transaction  tracking,  or  analysis  of  digital  libraries. 
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•  Wide  Area  Surveillance,  Intelligence  Report  Analysis,  and  Special  Events  have 
the  most  complex  goals  and  require  a  wide  range  of  data  types. 

•  The  VOI  Dossier  analysis  could  benefit  from  virtually  any  type  of  data  available. 

•  Almost  all  potential  applications  are  exploratory  -  the  RJOCs  need  to  explore 
their  domain  for  the  unexpected. 

•  All  the  potential  applications  involve  visualizing  time. 

Although  not  shown  in  the  table,  visual  analytic  tools  will  need  to  support: 

•  Schneiderman’s  standard  visual  analytic  sequence  (Section  2.2):  overview,  zoom, 
filter,  drill-down,  view  links,  show  history,  extract  information. 

•  Easy  methods  for  clearly  communicating  infonnation  to  the  Admiral. 

Predictions  and  “what-if  ’  analyses  are  also  important,  as  evidenced  by  Cdr  Renaud’s 
definition  of  Situation  Awareness: 

Situation  Awareness  is  the  cognitive  product  of  combining  the  operating 
environment  with  the  subjects  of  interest,  their  activities,  and  any  available 
intelligence,  to  infer  and  weigh  potential  outcomes  over  time.  ([105]  pg  5) 


Topical  Review  of  the  Literature 

The  literature  survey  was  split  into  the  topics  shown  in  Table  4-2  and  discussed  in 
Sections  4.3.1  through  4.3.1 1. 
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Table  4-1 

Maritime  Applications 
Cross-Referenced  to 
Goals  and  Data  Types 


•  Major  Requirement 
4  Minor  Requirement 
®  Possible  Future  Data  Type 


Ref.  Figure  2-24*  Ref  Task  2 
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Predict 

Scientifically  Investigate 
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Create  Structure 
Identify  Rare  Event 
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Assess  Competing  Hypotheses 
Collaborate 
Unstructured  Text 
Structured  Text 
Complex  Document 
Image 
Video 

Voice  Audio 
Sound  Audio 
Cyber  Transaction 
Finance  Transaction 
Shipping  Transaction 
Numeric  Values 
Graph  Structures 
Databases  /  Files 
Digital  Libraries 
Model  /  Simulations 
Geospatial  Coordinates 
Time 
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Uncertainty  /  Imprecision 
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Table  4-2 

Maritime  Applications  Cross- 
Referenced  to  Visual  Analytic  Topics 


•  Major  Requirement 
I  Minor  Requirement 
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4.3.1 

Geo-Spatial-Temporal  Awareness 

• 

• 

4.3.2 

Geo-Spatial-Temporal  Data  Mining 

• 

• 

4.3.3 

Geo-Spatial-Temporal  Uncertainty 

• 

• 

• 

• 

4.3.4 

Geo-Temporal  Coverage  Analysis 

• 

4.3.5 

Predictions  and  What-lf  Analysis 

• 

• 

• 

4.3.6 

Classical  Sensor  and  Network  Analysis 

• 

• 

• 

• 

• 

4.3.7 

Social  Network  Analysis 

• 

• 

• 

4.3.8 

Collaboration 

• 

• 

• 

4.3.9 

Textual  and  News  Feed  Exploration 

• 

• 

• 

4.3.10 

User  Interfaces  and  Usability 

• 

• 

• 

• 

4.3.11 

Visual  Analytic  Design  Principles 

.1  Geo-Spatial-Temporal  Situation  Awareness 

This  section  surveys  visual  analytic  strategies  that  help  analysts  be  aware  of  everything 
that  is  going  on  in  a  spatial  region  of  interest,  as  a  function  of  time,  including  “to  infer 
and  weigh  potential  outcomes  over  time”  [105],  Note  that  not  all  the  papers  have  an 
analytical  component  -  some  are  just  “Visualization.” 

Ship  Track  Analysis 

•  Andrienko  discusses  plotting  ship  tracks  and  multiple  tracks  ([3],  [4], [6]). 

•  Chan  discusses  how  to  make  the  display  responsive  even  when  accessing  very 
large  numbers  of  tracks  [23]. 

•  Willems  ([141-142])  describes  a  very  specific  3D-like  graphical  strategy  for 
visualizing  track  patterns  in  a  very  busy  part  of  the  North  Sea,  using  AIS  data. 
Tracks  are  interpolated  and  convolved  with  a  kernel  in  such  a  way  that  the 
accumulated  view  represent  ship  density,  rather  than  number  of  tracks  (see  Figure 
4-1).  Not  visual  analytics,  but  visually  effective. 
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Figure  4-1  Willem’s  Ship  Track  Rendering 

Willems  has  developed  a  striking  visualization  scheme,  shown  here  representing  ships  approaching  Amsterdam. 

(from  [142]).  The  smooth,  broad  strands  represent  average  ship  density,  and  the  superimposed  fine  ridges  represent 
individual  ship  tracks.  Raised  dots  are  thus  ships  at  anchor.  A  coloured  wash  is  superimposed  indicating  ship  density 
(ships  per  square  kilometre). 


Visualizing  Motion 

•  Aigner  [2]  has  a  very  helpful  presentation  reviewing  many  different  ways  to 
display  time,  and  periodic  events  in  particular. 

•  Few  [40]  uses  a  time-slider  and  leaves  trailing  marks  behind  to  visualize  change 
and  motion. 

•  Mehta  [87]  has  built  a  simple  set  of  inter-linked  windows  for  interactively 
visualizing  time-periodic  motion.  Not  as  useful  as  other  temporal  tools. 

•  Yang  [145]  describes  ideas  for  showing  moving  things  superimposed  on  static 
display  items. 

Dynamic  Non-Localizable  Events 

•  Maciejewski  [83]  discussed  visual  analytics  of  geo-spatially  distributed  features 
that  change  with  time,  which  may  be  relevant  to  the  time -varying  situational 
picture. 

•  Wright  [143]  proposes  “blobology”  to  represent  a  distributed  group  or 
meteorological  event,  including  the  time-evolution  of  the  shape  and  location  of 
that  group. 
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Coordinated  Multiple  Views 

•  Roberts  [110]  gives  an  overview  of  Coordinated  and  Multiple  Views  research, 
which  is  a  core  component  of  many  Visual  Analytics  products. 

Visualizing  Spatio-Temporal  Event  Patterns 

•  Word  clouds  used  to  illustrate  Tweet  traffic  during  the  superbowl  [13]  illustrate 
how  effective  and  intuitive  a  simple  time-slider  can  be.  This  should  be  viewed  as 
a  video,  or  better  yet  go  to  the  website  and  interact  with  it. 

•  Mehta  [87]  shows  very  simple  interactive  visualization  of  periodic  motion.  Not 
visual  analytics. 

•  Meiguins  [89]  demonstrates  a  spatio-temporal  model  of  one  of  the  VAST 
challenges,  with  link  visualizations. 

•  Ho  [54]  demonstrates  GeoAnalytics  using  coordinated  multiple-linked  views 
applied  to  a  large  ocean  space  volume  dataset. 

Kinematic  Anomaly  Detection 

•  Riveiro  [109]  describes  a  system  that  supports  the  acquisition  of  situation 
awareness  with  user  involvement  in  the  anomaly  detection  process  using  two 
layers  of  interactive  visualizations.  An  interactive  data  mining  module  supports 
the  insertion  of  the  user’s  knowledge  and  experience. 

•  Riveiro  et  al  [107]  provide  a  useful  discussion  of  the  cognitive  process  of 
anomaly  detection  based  on  observations  in  Ops  Centers. 

•  Hao  [53]  presents  a  way  to  visualize  anomalies  in  massive  streams  of  data. 

Visualizing  Threats 

•  The  International  Maritime  Bureau  publishes  a  Live  Piracy  Map  [68]  that 
illustrates  the  value  of  a  very  simple  situational  picture. 

•  The  UN  Institute  for  Training  and  Research  (UNITAR)  Operational  Satellite 
Applications  Programme  (UNOSAT)  [135]  uses  a  number  of  clear  visualization 
strategies  for  visual  data  mining  of  maritime  events,  as  shown  in  Figure  4-2. 


.2  Geo-Spatial-Temporal  Data  Mining 

This  section  looks  for  strategies  to  extract  patterns  from  past  behavior  in  a  dynamic 
geo-spatial  area  of  interest. 

•  Keim  [76]  outlines  the  advantages  of  visualization  for  data  mining  applications, 
and  gives  a  long  list  of  examples. 
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Figure  4-2  UNOSAT’s  Rendering  of  Pirate  Activity  Off  Somalia 

This  excerpt  from  a  custom-rendered  report  ([135])  illustrates  the  power  of  visual  renderings  for  threat  assessment. 
Red-tipped  cones  represent  the  density  of  pirate  attacks  in  the  Gulf  of  Aden  in  2008.  The  Maritime  Security  Patrol 
Area  (MSPA)  is  shown  as  a  darker  blue  band,  and  each  attack  that  occurred  within  that  band  are  noted  with  small 
“marker  pins.”  This  plot,  together  with  other  analyses,  showed  that  the  pirates  had  focused  their  attention  on  the 
MSPA  after  it  was  established  in  mid-2008. 


Ship  Normal  Routes  in  Time 

•  Andrienko  [4]  displays  methods  for  visualizing  average  speed  at  each  spatial 
location,  and  then  (more  interestingly)  unwitting  groups  of  vehicles  (or  vessels) 
that  follow  the  same  route  independently,  as  in  Figure  4-3. 

•  Chan  [23]  describes  an  architecture  for  keeping  a  visual  analytics  system 
responsive  even  when  indexing  massive  time  series  datasets. 

•  To  simplify  the  RMP  it  may  be  helpful  to  find  common  patterns  in  the  shapes  of 
ship  tracks,  using  an  automated  tool  to  indicate  whether  two  tracks  have  a  similar 
shape.  Schreck  [116]  looks  at  using  a  self-organizing  map  for  that  purpose.  This 
research  uses  Visual  Analytics  to  help  guide  the  self-organizing  map  to  create  the 
tool. 

Time  and  Location  Patterns  of  Past  Events 

•  Fails  shows  a  simple  multi-view  tool  that  highlights  when  extended  events 
overlap  in  time  [38], 

•  Jern  [70]  uses  multiple  coordinated  views  to  explore  spatio-temporal  patterns. 

•  Bak  [9]  has  an  original  mechanism  for  summarizing  the  timing  of  observed 
events. 

•  See  also  the  UNOSAT  [135]  report  mentioned  above. 
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cluster  12  (23) 
I-  cluster  23  (1 9) 
cluster  31  (19) 
cluster  37  (1 8) 
cluster  26  (1 5) 
cluster  29  (1 5) 
cluster  1 4  (1 0) 
cluster  1 5  (9) 
cluster  25  (9) 
cluster  27  (9) 
cluster  28  (9) 

VlvIanmulateT^^^M 

Figure  4-3  Andrienko’s  Clusters  of  Trajectories 

Some  “clusters”  of  cars  driving  onto  Milan  unwittingly  follow  the  same  route  as  many  other  cars.  Visual  Analytics  is 
here  being  used  to  identify  those  clusters.  In  a  maritime  domain,  they  might  be  clusters  of  ships  all  heading  for  a 
particular  port  of  seaway  (from  [4]). 


Data  Mining  Ship  Attribute  Data 

•  Hao  [52]  shows  how  to  transform  a  very  large  multi-attribute  time  series  into  a 
series  of  coloured  icons,  with  the  shape  of  the  icons  encoding  the  multiple 
attributes  so  that  changes  over  time  can  be  seen  visually. 

•  Wilkinson  et  a  I  [140]  suggests  that  we  need  visual  analytics  for  three  principal 
purposes:  checking  raw  data  for  anomalies,  exploring  data  to  discover  plausible 
models,  and  checking  model  assumptions.  This  paper  looks  at  finding  patterns  in 
very  high-dimensional  data,  using  statistical  transformations  to  re-project  the  data 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


4-11 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


mMDA 


and  thus  reveal  the  patterns,  and  plots  that  link  large  numbers  of  two-dimensional 
views  (usually  scatter  plots)  of  the  data. 

Making  Visual  Analytic  Systems  Better 

•  Sifer  [119]  defines  metrics  to  compare  interfaces  for  interactive  exploration  of 
hierarchical  multi-dimensional  data,  and  uses  those  metrics  to  evaluate  various 
visualization  strategies. 

•  Chen  [25]  lays  out  a  knowledge  management  methodology  that  should  (he 
suggests)  work  in  parallel  to  the  visualization,  as  part  of  the  visual  analytics. 


4.3.3  Geo-Spatial-Temporal  Uncertainty 

•  Jones  [72]  makes  a  serious  attempt  at  representing  uncertainty  in  a  maritime 
situation  map,  in  this  case  using  "squarified  treemaps."  Not  entirely  successful. 

•  Riveiro  [106]  looks  at  uncertainty  representations  in  a  number  of  fields,  from 
graphical  rendering  to  surveillance  displays,  but  does  not  come  up  with  definitive 
recommendations  about  how  it  should  be  done. 

•  Xie  [144]  describes  various  ways  to  visually  reveal  errors  or  data  quality 
problems  in  individual  numbers,  whole  records,  or  data  sources. 

•  Falkman  [39]  indicates  that  VISAD  is  engineered  to  represent  uncertainty,  but 
gives  no  details  about  how,  or  what  it  looks  like. 


4.3.4  Geo-Temporal  Coverage  Analysis 

There  is  no  apparent  work  being  done  to  develop  visual  analytics  for  sensor  coverage 
analysis. 

•  Matthews  [85]  proposed  a  shaded-area  approach  to  surveillance  coverage  maps, 
and  has  some  CF  analysts  test  them.  His  focus  was  on  visualizing  the  spatial 
extent  of  coverage,  but  not  its  senescence. 

•  Mewett  [90]  delivers  one  of  very  few  papers  that  looks  at  visualization  of  sensor 
coverage  patterns  over  time.  He  implemented  a  solution  in  which  coverage  areas 
fade  with  time,  to  indicate  that  the  data  is  stale.  Not  visual  analytics  yet,  but 
could  be  the  foundations  for  it. 
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Figure  4-4  Mewett’s  Display  of  Sensor  Coverage 

Black  lines  are  flight  lines  of  surveillance  aircraft,  and  yellow  patches  show  recent  sensor  coverage  from  the  aircraft, 
projected  onto  the  ground,  (from  [90])  This  is  neither  the  fanciest  visualization  nor  the  most  complicated  algorithm, 
but  it  is  so  rare  to  find  any  researchers  caring  about  sensor  swaths,  that  it  deserves  extra  visibility.  More  work  needs 
to  be  done  in  this  area. 


4.3.5  Predictions  and  What-lf  Analysis 


Predictions  within  the  Current  Plot 

•  Yue  et  a l  [146]  describe  an  AI  blackboard-based  agent  that  leverages  interactive 
visualization  and  mixed-initiative  problem  solving  to  enable  analysts  to  perform 
predictive  analysis  on  large  amounts  of  data. 

•  Sofrelog  [120]  provides  support  for  predicting  future  motion  of  a  ship. 

•  The  Thetus  Intelligence  Fusion  Platform  [127]  claims  an  ability  to  “Create  cause- 
and-effect  scenarios  to  determine  best  course  of  action,  plan  for  contingencies, 
and  maximize  resource  allocation,”  which  may  indicate  this  capability. 

Virtual  World 

Visual  analytics  may  one  day  take  the  fonn  of  very  detailed  multi-player  simulations  to 
extract  predictions  of  the  long-term  outcomes  of  various  tactical  choices.  Defence 
scientists  are  thus  interested  in  Virtual  World  research. 
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•  May  [86]  leads  a  panel  discussion  to  introduce  the  domain  of  visual  analytics  to  a 
virtual  reality  (VR)  and  artificial  reality  (AR)  audience  and  explore  how  and 
where  VR/AR  research  can  be  adapted  for  use  in  visual  analytics. 

•  Magazine  articles  [16]  [112]  and  [17]  provide  non-technical  overviews  of  how  the 
military  is  experimenting  with  virtual  worlds. 


4.3.6  Sensor  and  Network  Analysis 

There  are  very  many  papers  on  visualizing  digital  networks,  but  not  so  many  on  visual 
analytics  for  network  surveillance  or  network  analysis.  No  papers  were  found  that 
specifically  focused  on  sensor  functionality. 

Watching  for  Network  Failure  or  Intrusion 

•  Burner  [  1 9]  describes  First  Look. 

•  Mansmann  [84]  describes  a  visual  analytics  solution  for  network  monitoring. 

•  Meier  [88]  focuses  on  ways  to  visualize  network  behaviour  as  a  function  of  time, 
using  innovative  visual  patterns  as  shown  in  Figure  4-5. 


Figure  4-5  Meier’s  Visualization  of  Network  Time  Series 

Meier  focuses  on  how  network  information  (e.g.,  load  profiles,  port  information,  IP  packet  information,  etc.)  change 
with  time.  This  might  be  applicable  to  monitoring  of  sensor  networks. 
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Exploring  Node-Linked  Spaces 

•  Carlsen  [20],  [21]  describes  an  interactive  visualization  (almost  visual  analytics, 
but  lacking  the  ability  to  build  a  model)  that  was  created  for  an  art  project. 


4.3.7  Social  Network  Analysis 


Public  Social  Networking  (Twitter,  Facebook,  Wiki,  etc.) 

•  Barash  [10]  has  an  Excel  plug-in  for  analysing  social  networking  traffic.  This 
could  become  important  if  the  CF  adopts  social  networking  as  a  medium. 

•  A  delightful  dynamic  visualization  [13]  of  social  networking  during  the 
superbowl  illustrates  the  power  of  a  simple  concept  well  implemented. 

•  Suh  [126]  shows  how  visualization  and  visual  analytics  can  provide  important 
insights  into  Wiki-type  collaborative  knowledge  bases  (such  as  Intellipedia  for 
example). 

Social  or  Vocational  Relationships 

•  Godwin  et  z?/  [51]  demonstrates  how  gene-sequencing  algorithms  can  be  used 
interactively  to  identify  patterns  of  collaboration  among  criminals. 

Time  Evolution  of  Networks 

•  Kapler  [73]  proposes  “Configurable  Spaces”  as  a  way  to  view  the  time-evolution 
of  arbitrary  and  abstract  network  maps. 


4.3.8  Collaboration 

•  Brennan  [15]  clearly  describes  a  collaboration  architecture  that  appears  to  be 
suitable  for  jointly  developing  the  Maritime  Picture. 

•  Hollan  [55]  suggests  a  new  “distributed  cognition”  paradigm  is  needed  to 
thinking  about  human-computer  interaction  in  a  distributed  visual  analytic 
system. 

•  Bowman  et  al  [14]  lay  out  management  methodologies  and  team  interaction 
standards  to  be  used  when  scientific  teams  from  different  agencies  and  different 
allied  countries  have  to  work  together  intensely. 

•  Keel  [75]  describes  a  comprehensive  and  infonnation-rich  report  on  a 
collaboration  architecture  "EWall"  where  people  engaged  in  the  visual 
organization  of  information  using  graphical  objects  (Cards)  share  their  work  via 
software  agents. 
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•  Vernik  and  Bouchard  [136]  describe  Imago,  a  distributed  environment  that 
supports  visualization,  and  thus  a  candidate  for  hosting  collaborative  visual 
analytics. 

•  Weaver  [139]  suggests  collaboration  can  be  achieved  by  giving  separate  users 
coordinated  and  linked  views  of  the  analysis  space,  and  giving  managers  tools  to 
"metavisualize"  the  data. 

•  Robinson  [111]  ran  experiments  to  watch  how  people  collaborate,  and  then  built 
an  architecture  to  support  the  various  approaches  that  they  used. 

•  The  Starlight  video  [101]  suggests  that  it  supports  collaboration,  but  by 
“collaboration”  it  seems  to  mean  either  a)  two  people  sitting  at  the  same  tenninal 
or  b)  using  starlight  graphics  to  brief  a  larger  group. 


Textual  and  News  Feed  Exploration 

•  David  Fisher  [42]  has  built  a  text- analysis  tool  that  looks  for  themes  and 
associated  keywords,  and  then  tracks  how  those  keywords  evolve  over  time. 

•  Luo  et  al  [82]  have  built  a  prototype  news  analysis  system  that  identifies  news 
clips  by  topic  and  provides  a  visual  exploration  medium. 

•  Ghoniem  et  al  [49]  use  a  Theme-River- like  graphical  portrayal  to  show  how 
topics  in  the  news  (gleaned  from  the  close  captions  of  all-news  television  stations) 
vary  with  time. 


User  Interfaces  and  Usability  for  Visual  Analytics 

•  Butkiewicz  et  al  [18]  found  the  multi-touch  table  to  be  very  popular  and  intuitive. 


Visual  Analytic  Design  Principles 

•  Andrienko  et  al  [5]  lay  out  a  manifesto  for  “GeoVisual  Analytics”  as  a  sub-type 
of  visual  analytics. 

•  Pere  and  Schneiderman  [103]  discuss  the  tight  link  between  visual  analytics  and 
data  mining. 

•  Betini  [12]  suggests  that  data  mining  and  data  visualization  need  to  be  better 
coordinated. 

•  Chaomei  Chen  [24]  uses  information  theory  to  guide  visual  analytic  systems  so 
that  the  visualizations  are  not  deceptive. 


4-16 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


BMDA 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


4.4  Guide  to  the  Bibliography 

The  annotated  bibliography  is  attached  as  Appendix  B.  It  lists  all  the  papers  and  web 
sites  that  were  collected,  examined,  and  referenced  in  the  text. 


4.4.1  The  EndNote  Database 

A  total  of  213  references  were  inserted  into  the  EndNote  database,  as  shown  in  Figure 

4-6.  References  were  collected  systematically  as  follows: 

•  Searches  were  done  on  the  phrase  “visual  analytics”  in  IEEExplore,  SPIE  Digital 
Library,  and  ACM  Portal,  and  the  abstracts  for  every  paper  found  in  those  three 
databases  were  examined.  Papers  on  topic  for  this  study  were  then  downloaded 
and  scanned  to  confirm  relevance  before  being  included. 

•  Searches  were  done  using  Google  and  corporate  links,  for  the  web  pages  of  every 
regional  VAC  member,  Center  of  Excellent,  or  FODAVA  partner.  On  those  web 
pages,  references  to  relevant  technology  were  followed  up  if  possible. 

•  Searches  were  done  on  Google  for  “visual  analytics”  and  “maritime”  with 
“analytics”  plus  other  searches  on  specific  topics  such  as  “visual”  and 
“collaboration.” 

Almost  all  the  relevant  references  were  from  the  past  4  years,  as  shown  in  Figure  4-7. 


Figure  4-6  Scope  of  the  Literature  and  Product  Search 

The  literature  search  focused  mainly  on  visual  analytics  papers  (95  were  collected  in  the  Endnote  database)  and  on 
visual  analytic  products  (49  were  collected).  The  database  also  includes  a  total  of  47  references  to  visual  analytics 
people,  groups,  conferences,  and  periodicals. 
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Figure  4-7  Age  of  the  References 

Almost  all  of  the  visual  analytics  references  in  the  EndNote  database  were  published  since  the  term  “visual  analytics” 
was  established  by  Thomas’s  2005  book  “Illuminating  the  Path”  [131].  There  are  213  references  to  papers, 
institutions,  and  relevant  websites,  144  of  which  are  listed  in  the  annotated  bibliography. 


4.4.2  Annotated  Bibliography 

The  Annotated  Bibliography  in  Appendix  B  lists  those  items  in  the  database  that  are 
particularly  relevant  to  the  Maritime  Domain. 

As  shown  in  Figure  4-8,  the  bibliography  provides  standard  citation  information,  plus 
our  assessment  of  what  is  valuable  in  the  article,  product  description,  conference,  or 
group  website.  If  a  copy  of  the  article  was  delivered  with  this  report,  the  fdename  is 
also  listed. 


Citation  Information 
. \ . 


Reference  Number  Citation  Information  Web  Link 

- ^ . . - . 

[113]  J.  Stasko,  C.  Gorg,  and  Z.  Liu.  "Jigsaw:  supporting  investigative  analysis  through  interactive 
visualization,"  Information  Visualization,  vol.  7,  pp.  1 18-132,  http:// ww w, pal grave- 
iournals.com/ivs/iournal/v7/n2/pdf/9500l  80a.pdf.  2008.  [4  copy  is  attached  as  file: 

StascoJigsaw.pdf] 

A  good/ournal  article  describing  Jigsaw,  including  a  frank  discussion  of  some  of  its 
limitations  (e.g.  Jigsaw  does  not  provide  overviews  that  help  an  analyst  find  something 
interesting  to  start  looking  at. 


File  Name  on  Disk 


Comments  on  the  Paper 


Figure  4-8  Example  Bibliography  Entry 

The  bibliography  provides  standard  citation  information  such  as  Author(s),  Title,  Journal  or  Conference  name, 
Location,  and  Date.  If  possible,  it  also  provides  a  web  link  from  which  the  paper  can  be  acquired.  If  a  copy  of  the 
paper  is  included  on  the  distribution  disk,  the  name  of  that  file  is  also  provided.  Our  assessment  of  the  paper  is 
summarized  in  a  separate  paragraph  immediately  following  the  citation,  in  italics. 
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PRODUCT  REVIEW 


Table  5-1  through  Table  5-15  summarize  a  number  of  visualization  and  visual  analytic 
“products”  that  were  assessed.  In  accordance  with  the  contract  scope,  the  products 
were  not  installed  and  tested.  Each  table  provides  a  visual  summary  of  the  product, 
often  in  the  fonn  of  a  screen  grab,  plus  a  written  description  of  what  the  product  does. 
The  question  of  whether  the  product  is  truly  “visual  analytics”  is  addressed  by 
tabulating  its  conformance  to  the  first  five  rules  in  (2-2)  through  (2-6).  References  are 
provided  for  each  product,  and  if  a  video  was  available  for  download,  the  filename  of  an 
.avi  file  is  provided. 

Geographic  Analytics 

The  strengths  and  weaknesses  of  the  seven  software  systems  that  are  particularly  strong 
at  visualizing  geo-spatial  infonnation  can  be  summarized  as  follows: 

•  Starlight:  (Table  5-1)  Excellent  Visual  Analytics  credentials  and  capabilities.  It 
does  provide  geo-spatial  support  but  most  of  its  energy  is  focused  on  text  and  link 
analysis.  No  indication  of  true  support  for  collaboration.  User  interface  is 
perhaps  too  complex  for  RJOC  analysts  to  use  routinely. 

•  GeoTime:  (Table  5-2)  Very  good  Visual  Analytics  credentials,  but  focused  on 
geo-spatial-temporal  renderings  and  hence  a  narrower  visualization  range.  RJOC 
leaders  liked  its  appearance,  but  thought  it  might  be  too  complex  for  routine 
analysis.  Not  collaborative. 

•  Palanterra:  (Table  5-3)  Apparently  not  very  visually  sophisticated  and  hence  not 
really  Visual  Analytics.  But  has  good  collaboration. 

•  ENVI  iV-Dimensional  Visualizer:  (Table  5-4)  a  true  visual  analytics  tool, 
deployed  long  before  Visual  Analytics  was  a  recognized  term,  but  focused  very 
specifically  on  ground  classification  and  target  detection  and  hence  a  very  narrow 
visualization  range.  Not  collaborative. 
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•  VISAD:  (Table  5-5)  a  true  Visual  Analytics  tool  that  already  supports  maritime 
surveillance.  There  is  some  concern  that  the  visualizations  are  too  conceptual  and 
thus  it  might  not  be  popular  in  the  RJOCs. 

•  GeoAnalytics  Visualization  (GAV):  (Table  5-6)  an  excellent  collection  of 
sophisticated  visualizations,  with  good  support  for  exploration  and  discovery,  but 
no  apparent  support  for  analytics  (i.e.  building  a  case). 

•  Command  Post  of  the  Future:  (Table  5-7)  highly  interactive,  analytical,  and 
collaborative,  with  good  visualizations.  But  the  visualizations  appear  to  be  much 
less  sophisticated  than  classic  Visual  Analytics. 

•  Sofrelog:  (Table  5-8)  state-of-the-art  commercial  application  for  displaying  the 
maritime  picture,  includes  some  support  for  collaboration  and  predicting  future 
tracks.  Lacks  enough  visual  sophistication  to  be  true  Visual  Analytics. 

Textual  Analytics 

The  following  five  systems  are  all  focused  mainly  on  analyzing  database  records  and 

textual  data, 

•  In-Spire:  (Table  5-9)  with  a  truly  impressive  list  of  visualizations  and  Visual 
Analytics  features,  the  only  concern  here  is  that  the  user  interface  may  be  difficult 
and  slow  to  learn.  The  Assessment  Wall  ([99])  is  an  attempt  to  reduce  that 
complexity. 

•  Jigsaw:  (Table  5- 10)  a  very  powerful  Visual  Analytics  tool  that  is  popular  with 
analysts  because  it  is  very  simple  and  intuitive  to  use. 

•  Analyst’s  Notebook:  (Table  5-1 1)  seems  to  be  moving  from  being  primarily  a 
visualization  and  report-generating  tool  toward  true  “analytic”  capability  where  a 
storyline  is  crafted  by  the  analyst.  This  has  been  installed  in  the  RJOCs  for  many 
years  but  is  not  used  regularly  because  it  takes  too  long  to  get  the  data  into  the 
system  (and  much  of  the  interesting  non-tracking  data  is  behind  firewalls). 

•  Visualinks:  (Table  5-12)  appears  to  be  trying  to  compete  with  Analyst’s 
Notebook  but  with  even  less  “analytic”  capability.  They  mention  collaboration 
but  it  sounds  like  a  product-sharing  capability  rather  than  an  “analysis  process” 
sharing  capability. 

•  Thetus  Intelligence  Fusion:  (Table  5-13)  uses  similar  data  and  works  toward 
similar  goals  as  Analyst’s  Notebook  and  Visualinks.  The  major  difference  is  that 
they  have  a  clear  analysis  track,  using  an  embedded  ontology  and  visualization, 
by  which  the  analyst  can  build  understanding. 

Knowledge  Mapping 

•  K-Mapper:  (Table  5-14)  a  sophisticated  tool  for  exploring  and  discovering 
people,  expertise,  assets,  and  concepts.  But  it  does  not  yet  support  interactive 
analysis  -  the  building  of  the  analytical  “story.” 
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Interface  Hardware 

•  3D  Touch  Table:  (Table  5-15)  very  impressive  ability  to  “display”  terrain  in  3D. 
Probably  not  enough  added  value  for  the  RJOCs  to  acquire  one. 

•  Assessment  Wall:  (Table  5-16)  is  a  large-format  touch-sensitive  display 
engineered  to  work  with  In-Spire  software. 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 
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Table  5-1  Starlight 


Starlight  is  perhaps  the  definitive  Visual  Analytics  system.  It  provides  advanced  information 
visualization  technology  for  graphically  manipulating,  understanding,  and  acting  on  very  large  quantities 
of  complex  and  disparate  information.  Features:  Advanced  search  and  data  filtering  capabilities; 
Simplified  data  acquisition;  Visual  information  analysis  environment;  Network  diagrams;  Free-text  topic 
clustering;  Geospatial  relationships;  Temporal  relationships;  Hierarchical  organizations  or  collections  of 
information;  Categorical  relationships;  Multidimensional  correlation  of  structured  data;  Advanced 
workflow  management  capabilities;  Information  fusion  among  disparate  data  sets;  Multimedia  data 
support.  Currently  Starlight  is  not  set  up  for  real-time  operation. 


Visual  Soph:  Y 


Interactive:  Y 


Exploratory:  Y 


Analytical:  Y 


Collab:  N? 


Potential  Maritime  Application:  VOI  Dossier  and  Track,  Intellipedia  Connections,  Knowledge 
Indexing,  Text  in  Intelligence  Reports,  Special  Event 


Vendor:  Future  Point  Systems  Inc. 


Video:  Starlight_video.avi 


References:  [44]  and  [45]  and  [101]  and[114] 


5-4 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


BMDA 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


Table  5-2  GeoTime 


GeoTime  was  developed  to  improve  perception  of  movements,  events  and  relationships  as  they  change 
over  time  within  a  spatial  context.  A  combined  temporal-spatial  space  was  constructed  in  which  to  show 
interconnecting  streams  of  events  over  a  range  of  time  in  a  single  picture.  Events  are  represented  within 
an  X,  Y,  T  coordinate  space,  in  which  the  X,  Y  plane  shows  geographic  space  and  the  Z-axis  represents  time 
into  the  future  and  past.  In  addition  to  providing  the  spatial  context,  the  ground  plane  marks  the  instant  of 
focus  between  before  and  after;  Events  along  the  timeline  “occur”  when  they  meet  the  surface.  Events 
are  arrayed  in  time  along  time  tracks,  which  are  located  wherever  events  occur  within  the  spatial  plane. 


Visual  Soph:  Y/N 

Interactive:  Y 

Exploratory: 

Y 

Analytical:  Y 

Collab:  N? 

Potential  Maritime  Application:  VOI  Dossier  and  Track,  What  is  Normal  Behaviour,  Wide  Area 

Surveillance,  Surveillance  Coverage,  Special  Event 

Vendor:  Future  Point  Systems  Inc. 

Video:  Geotimel.avi 

References:  [74]  and  [97] 
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Table  5-3  Palanterra 


Built  using  commercial-off-the-shelf  software,  Palanterra  supports  thick-  and  thin-client  operations. 
Accessing  Palanterra’s  national-level  site  provides  critical  infrastructure  and  asset  data  for  the  entire 
United  States.  Zooming  to  an  urban  site  gives  controlled  access  to  high-resolution  datasets  of  major 
cities.  HSIP  data  are  vector-  and  raster-based  and  include  satellite  and  airborne  imagery  and  light 
detection  and  ranging  (LIDAR)  data.  Such  products  are  employed  to  create  virtual  analytic  environments 
for  3D  analysis  in  support  of  national  security  events.  One  example  is  the  use  of  such  3D  models  for  line- 
of-sight  analysis  to  ensure  security  at  the  2002  Super  Bowl  in  San  Diego,  California. 


Visual  Soph:  N 

Interactive:  Y 

Exploratory:  Y 

Analytical:  N 

Collab:  Y? 

Potential  Maritime  Application:  Special  Event 

Vendor:  US  NGA 

Video:  none 

References:  [11] 
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Table  5-4  ENVI  iV-Dimensional  Visualizer 


S  elect  Plot  ▼  Clear  Plot 


Basic  Stats 
Band  1 
Band  2 
Band  3 
Band  4 
Band  5 
Band  6 
Band  7 
Band  8 
Band  9 


Min 

1235 

1885 

2164 

2237 

2348 

2319 

2309 

2367 

2284 


Max 

2249 

3319 

3677 

3645 

3737 

3649 

3625 

3671 

3528 


Me, 

1769.10001 

2680.30001 

3036.0806. 

3063.35801 

3099.7548: 

3069.1064! 

3043.1322! 

3066.5354! 

2965.9483! 


Available  Bands  List 


E 

File  Options 


Fl  gll  cup99_swir 

□  Resize  (Band  95: cup 

□  Resize  (Band  96:cup 

□  Resize  (Band  97:cup 

□  Resize  (Band  98: cup 

. □  Resize  (Band  99: cup 

;  □  Resize  (Band  100:cu 

®o  Resize  (Band  101:cu 
□  Resize  (Band  1 02: cu 

. □  Resize  (Band  103:cu 

. □  Resize  (Band  1 04: cu 

. □  Resize  (Band  105:cui 


n-D  Controls 


File 


Class 


options 


n-D  Selected  Bands 

iwa  3 


S  peed  [35  View  1 10/47 


Start 

Step  <-  j  ->  J  New 

f  Gray  Scale  (•  RGB  Color 


File  ROI_Type  Options  Help 


Window:  (*  Image  C  Scroll  C  < 


ROI  Name  Color 

Road  Red 

12 

x 

■  Green 

31 

© 

New  Region 


Select  All 


Hide  ROIs 


Show  F 


EH 


i :  lii^rnnimy 


,n  x 


File  Options 

Min/Max/Mean:Shale  [Green] 


12.1  2.2  2.3 

w  Wavelength 


Filename:  C:\Documents  and  Settings'^.) 
ROI :  Shale  [Green]  310  points 


The  /V- Dimensional  visualizer  (NDV)  was  engineered  about  ten  years  ago  for  extracting  information 
from  hyperspectral  images.  The  operator  opens  an  M- band  hyperspectral  image  (a)  and  uses  the  NDV 
controls  (b)  to  select  a  subset  of  bands  (e.g.  N=  4).  The  N-D  Visualizer  (c)  slowly  rotates  an  N- 
dimensional  scatter  plot  in  the  window.  If  the  operator  sees  a  cluster,  she  can  stop  the  rotation,  circle 
some  scatter  plot  points,  and  paint  them  a  selected  colour  (red  and  green  in  this  example).  This  creates 
corresponding  regions  of  interest  (d)  which  can  be  named  by  the  analyst,  and  it  marks  with  the  correct 
colour  all  pixels  in  the  geospatial  scene  with  matching  spectra  (e).  Note  how  the  red  cluster  has  identified 
the  road.  Statistics  can  be  calculated  for  each  ROI  showing  the  mean  spectrum  (f)  and  standard 
deviation.  Conversely,  on  operator  can  select  a  region  in  the  geospatial  scene  and  ask  for  all  pixels  in  the 
NDV  to  be  painted  that  colour,  thus  identifying  similar  pixels.  The  operator  thus  moves  between  four 
coordinated  views:  spatial,  scatter  plot,  spectral,  and  regional. 


Visual  Soph:  Y/N 

Interactive:  Y 

Exploratory: 

Y 

Analytical:  Y 

Collab:  N 

Potential  Maritime  Application:  What  is  Normal  Behaviour,  Sensor  Datastream,  Intellipedia 

Connections,  Surveillance  Coverage. 

Vendor:  ITT 

Video:  EnviForestry.avi 

References:  [69] 
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Table  5-5  VISAD 
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VISAD  features  include: 

•  Interaction  methods  that  allow  the  involvement  of  the  user  in  the  fusion  process,  data  exploration  and 

knowledge  discovery; 

•  Visualization  of  large  data  sets:  patterns,  trends,  changes  over  time,  outliers,  exceptions,  anomalies, 

relationships,  correlations,  clusters,  groups,  context,  etc; 

•  Visualization  of  uncertainty,  reliability,  completeness  and  quality  of  information; 

•  Visualization  of  different  levels  of  abstraction  or  granularity  (in  time  and  space). 

One  initial  application  is  maritime  anomaly  detection  and  situation  awareness. 


Visual  Soph:  Y/N 

Interactive:  Y 

Exploratory:  Y 

Analytical:  Y 

Collab:  N? 

Potential  Maritime  Application:  What  is  Normal  Behaviour,  Sensor  Datastream,  Intellipedia 

Connections,  Wide  Area  Surveillance,  Surveillance  Coverage. 

Vendor:  University  of  Skovde 

Video:  None 

References:  [39]  [108] 

5-8 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


BMDA 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


Table  5-6  GeoAnalytics  Visualization  (GAY) 


Volume  Viewer 


3D  Volume  Time 
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Enhanced  Scatter 
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Shaded 
Time  Graphs 
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Colour  Tube 


Dynamic 

legend 


GAV  is  a  visualization  toolkit  with  components  that  support  a  mixture  of  technologies  from  the  three 
data  visualization  fields:  information  visualization,  geovisualization,  and  scientific  visualization.  GAV 
renders  the  visualizations  using  Flash.  It  has  a  layered-components  architecture,  multiple-linked  and 
coordinated  views,  dynamic  classified  focus  and  context  maps,  and  parallel  coordinates  plot.  Although  it 
provides  complex  visualization,  it  may  not  be  true  visual  analytics  because  (based  on  the  descriptions  in 
the  papers)  it  does  not  allow  the  operator  to  insert  insights  into  an  emerging  model. 


Visual  Soph:  Y 

Interactive:  Y 

Exploratory:  Y 

Analytical:  N? 

Collaborative:  N? 

Potential  Maritime  Application:  VOI  Dossier,  What  is  Normal  Behaviour,  Sensor  Datastream, 

Intellipedia  Connections,  Surveillance  Coverage,  Special  Event 

Vendor:  University  of  Skovde 

Video:  None 

References:  [54,70-71] 
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Table  5-7  Command  Post  of  the  Future 


The  Command  Post  of  the  Future  is  an  executive  level  decision  support  system  providing  situational 
awareness  and  collaborative  tools  to  support  decision  making.  This  tool  suite  is  built  using  General 
Dynamics  C4  Systems  CoMotion  technology.  CoMotion  supports  visualization,  information  analysis, 
and  collaboration,  in  a  single,  integrated  environment  that  ultimately  helps  commanders  and  decision 
makers  analyze  information,  share  thoughts,  and  evaluate  courses  of  action. 

•  Team  members  collaborate  to  create  a  multi-perspective  shared  operational  picture. 

•  Supports  parallel,  synchronous  and  asynchronous  cross  functional  planning  and  execution. 

•  Allows  analysts  to  tailor  visualizations  to  suit  how  they  absorb  information,  thus  allowing  others  to 

see  what  they  are  thinking. 


Brochures  suggest  that  it  supports  a  relatively  small  group  of  visualization  options,  which  would  be 
appropriate  for  its  goals  and  end-user  needs. 


Visual  Soph:  N 

Interactive:  Y 

Exploratory:  Y 

Analytical:  Y 

Collaborative:  Y 

Potential  Maritime  Application:  What  is  Normal  Behaviour,  Wide  Area  Surveilance,  Surveillance 

Coverage,  Special  Event 

Vendor:  General  Dynamics 

Video:  None 

References:  [46] 
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Table  5-8  Sofrelog 


Superimposed  raw  radar  images  from  multiple  radars  are  displayed,  automatically  selects  the  best  sensor 
input  to  track  each  vessel,  shadows  for  one  sensor  are  automatically  filled-in  by  other  sensors. 
Customizable  event  alarm  structure.  The  entire  tracking  history  of  suspicious  vessels  can  be  displayed, 
showing  place  of  origin,  other  vessels  encountered.  Future  vessel  position  can  be  predicted  based  on 
current  course  and  speed.  Simulator  and  Training  modes  using  archived  incidence  from  your  area. 


Visual  Soph:  N 


Interactive:  Y 


Exploratory:  Y 


Analytical:  ? 


Collaborative:  Y 


Potential  Maritime  Application:  Wide  Area  Surveillance,  Surveillance  Coverage,  Special  Event 


References:  [120] 


Vendor:  Sofrelog 


Video: 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
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Table  5-9  In-Spire 


£2  Galaxy  -  EP3  (import) 


April  Middle  East  triage :  Editing 


;+T. 


■v  - 


Outliers 


Select:  Click  o 
Ctl- click  or  -d: 


[^j  Outrier  Terms  4 


Evidence  Summary 


▼  Normalize 


3  Document  Viewer  -  April  news  st...|_J[nJ[x] 


anded  on  is.  Crasned  Aliens  abduc. 

ncMqRnp' 


In-Spire  is  able  to  uncover  relationships,  trends,  and  themes  hidden  within  textual  data,  and  thus  create 
new  knowledge  and  new  insights  that  could  be  used  to  assess  terrorist  threats,  determine  how  to  treat  a 
medical  condition,  or  gather  market  research  on  the  competition.  It  provides  visual  interfaces  for: 
reading  similar  documents  together,  removing  less  interesting  clumps,  finding  relevant  information, 
keeping  track  of  what  is  found,  inspecting  non-hits  among  the  hits,  seeing  time  patterns,  seeing 
correlations  among  groups,  doing  triage,  managing  evidence  and  hypotheses,  summarizing  evidence, 


Visual  Soph:  Y 

Interactive:  Y 

Exploratory:  Y 

Analytical:  Y 

Collaborative:  N? 

Potential  Maritime  Application:  VOI  Dossier,  What  is  Normal  Behaviour,  Intellipedia,  Knowledge 
Indexing,  Text  in  Intelligence  Reports,  Special  Event 

Vendor:  Pacific  Northwest  National  Laboratory 

Video:  InSpire.avi 

References:  [100]  [98] 
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Table  5-10  Jigsaw 


>«IMH  DM 
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1 

-  « 

Jigsaw  is  a  suite  of  interconnected,  cooperating,  interactive  visualizations  that  present  multiple 
perspectives  on  a  document  collection  and  shows,  in  particular,  connections  between  entities  across  the 
collection.  It  uses  a  fundamentally  simple  user  interaction  model  in  its  views  that  make  the  system  less 
complex  for  analysts  to  use.  It  is  a  working  system  that  has  already  exhibited  investigative  value, 
winning  the  2007  VAST  Challenge  contest  ([47]).  Views  include:  List  view  (a)  Graph  View  (b),  Scatter 
Plot  View  Document  View,  Calendar  View,  Document  Cluster  View,  and  Shoebox  for  evidence 
marshalling. 


Visual  Soph:  Y 

Interactive:  Y 

Exploratory:  Y 

Analytical:  Y 

Collaborative:  N? 

Potential  Maritime  Application:  VOI  Dossier,  What  is  Normal  Behaviour,  Intellipedia,  Knowledge 
Indexing,  Text  in  Intelligence  Reports,  Special  Event 

Vendor:  Pacific  Northwest  National  Laboratory 

Video:  jigsaw_video.avi 

References:  [47,  124] 
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Table  5-11  Analyst’s  Notebook 


J  New  Standard  Chart 
New  From  Template... 
£  Add  Chart  Items 
J*  Import  Data 


V  Flters  and  Histograms 
^  Conditional  Formatting 
•I*.  Social  Network.  Analysis 
Map  Chart  Items 
Find  Cluster  Setup. . . 
link  Connectivity  Ousters 


Visual  Search. . . 
<3  Find  Path... 

!'**  Find  linked... 
*"]  list  Items... 


Analyst’s  Notebook  is  engineered  to  help  analysts  quickly  turn  large  sets  of  disparate  data  into  high 
quality  and  actionable  intelligence  to  prevent  crime,  fraud  and  terrorism.  Capabilities  include:  create 
charts  manually  or  automatically  from  structured  data  sources  including  databases,  online  data, 
spreadsheets;  visualize  and  analyze  (a)  large  volumes  of  raw,  multi-format  data  to  understand  the  who, 
what,  when,  where  and  why;  view  social  patterns  (b)  and  trends  unfolding  over  time  and  hone  in  on  key 
players  and  activities;  quickly  identify  and  understand  key  individuals  in  complex  target  networks; 
visualize  geographical  information  (c),  track  movements  of  suspects  over  time,  and  communicate 
complex  cases  using  intuitive  briefing  charts  that  capture  and  organize  supporting  data  to  tell  the  story. 


Visual  Soph:  Y 


Interactive:  Y 


Exploratory:  Y 


Analytical:  Y/N 


Collaborative:  N? 


Potential  Maritime  Application:  VOI  Dossier,  What  is  Normal  Behaviour,  Intellipedia,  Knowledge 
Indexing,  Text  in  Intelligence  Reports,  Special  Event 


Vendor:  12  Inc. 


Video: 


References:  [58] 
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Table  5-12  VisuaLinks 
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VisuaLinks  is  a  platform- independent,  graphical  analysis  tool  used  to  expose  hidden  activity  and 
behavioral  patterns  in  any  number  and  type  of  data  sources.  It  uses  graphical  representations  to  uncover 
underlying  relationships  and  patterns.  VisuaLinks  addresses  the  entire  analytical  process  -  query  and 
display  large  amounts  of  data  with  no  size  limit,  use  one  of  many  query  algorithms  to  expose  networks, 
patterns  and  trends  in  the  data,  visually  reveal  direct  and  indirect  linkages  within  the  data,  perform 
proactive  and  reactive  analysis  supporting  tactical  and  strategic  operations,  collaborate  with  peers  sharing 
data  and  related  analysis,  prepare  and  print  final  analysis  presentations  including  on  Google  Earth. 


Visual  Soph:  Y 

Interactive:  Y 

Exploratory:  Y 

Analytical:  Y 

Collaborative:  N? 

Potential  Maritime  Application:  VOI  Dossier,  What  is  Normal  Behaviour,  Intellipedia,  Knowledge 

Indexing,  Text  in  Intelligence  Reports,  Special  Event 

Vendor:  Visual  Analytics  Inc. 

Video: 

References:  [138] 
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Table  5-13  Thetus  Intelligence  Fusion  Platform 


Thetus  Diagrammer 

i*  A  □  X  A  il  P 


(a) 
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Columbus  2/0 

Grove  City 
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’I’MUr'SIJSb1  J  iWiiiwiyr1 


OHIO 

Janesville 


Relationships 


I  create  new  |  search  |  apply 


Ontology  Tree 

S’  c  Fusion 
i  ,c  Person 

C>  Suspect 

5  Cl  LawEnforcementOfficer 
0  c>  Criminal 

C>  Terrorist 
gi  vehicle 

6  NoteworthyEvent 

s  gBumasa 

0  c(  CriminalActivity 

(c) 


s 


Concept  Cloud 

Sorting:  Alphabetically  |  Popularity 

Drop  an  item  here  to  create  a  new  tag.., 

Alert  BaseClientConcepts  Burglary 

I  Car  CO!  COITask 

;  ControlledSubstancePurchase  Criminal 
CriminalActivity  FertilizerPurchase 
I  FlnancialTransaction  Firearm sPurchase 
Suspect  heavyTruck  Hoinldde 
-  IntelligenceAgent  SuspiclousActVity 
LawEnfcrcementOfflcer  licensed  Vehicle 
lightTruck  MapElement  truck 
I  Point  Police  NoteworthyEvent 
I  SuspiciousActivity  truck 
I  Vandalism  Vehicle  ViolentCriminal 


event  J_oc3tion 


contracted  from 


(b) 


(d) 


The  Thetus  Intelligence  Fusion  Platform  is  engineered  to  derive  integrated  intelligence  from  multi¬ 
source,  multi-format  data.  Automated  workflow  components  increase  the  speed  and  efficiency  of 
accessing  the  data,  building  an  interpretive  “story,”  which  may  include  a  network  diagram  (a),  a  map  (b), 
an  ontology  (c),  or  a  concept  cloud  (d),  and  automatically  assembling  and  publishing  reports  to  specific 
stakeholders  in  a  wide  variety  of  formats.  Users  can  create  and  analyze  cause-and-effect  scenarios  in 
support  of  proactive  threat  identification  and  interdiction.  Participating  communities  can  search,  access, 
and  share  information  using  the  familiar  language  and  context  of  their  discipline  while  maintaining 
shared  meaning  across  systems.  Intelligence  products  and  analysis  processes  are  preserved  for 
collaboration,  reusability,  and  ongoing  information  enhancement.  Collection  management  and 
dissemination  across  the  entire  system  are  restricted  by  policy  controls. 


Visual  Soph:  Y 

Interactive:  Y 

Exploratory:  Y 

Analytical:  Y 

Collaborative:  N? 

Potential  Maritime  Application:  VOI  Dossier,  What  is  Normal  Behaviour,  Intellipedia,  Knowledge 

Indexing,  Text  in  Intelligence  Reports,  Special  Event 

Vendor:  Thetus  Inc. 

Video: 

References:  [127-128] 
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Table  5-14  K-Mapper 


3  Tcmparatur... 
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- 
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3  Mort4r  Att- 

3  Deiowtof 
)  Fire 

1  Mortar  Att... 

>  Dynamite 
'I  Militarv  V... 

KMapper  provides  a  navigable  visualization  of  an  extended  information  space,  including  human 
resources,  knowledge  assets,  procedural  information,  and  conceptual  information.  It  is  thus  a 
sophisticated  and  interactive  visualization  supporting  exploration.  The  tool  does  not  currently  offer  any 
capability  for  an  analyst  to  build  a  model  or  tell  analytical  “story”  so  this  does  not  qualify  as  visual 
analytics  according  to  rule  (2-5). 


Visual  Soph:  Y 

Interactive:  Y 

Exploratory:  Y 

Analytical:  N 

Collaborative:  N 

Potential  Maritime  Application:  Knowledge  Indexing 

Vendor:  DRDC  Valcartier 

Video: 

References:  [81] 

3  Bomb 

» Ionizing  r... 

I  Response  T... 

» Emergency 

•  Emergency,.  * 

■ 

1  emergency ... 

udeir  M  Gary  With. 


J’KUStUfoh 


John  Coope... 


Frank  Estr. 


ft*' 


NBCRRT 


V»  1 

Canada  Bor.. 

"fr  Transport... 


)  Terrorist 
I Policy  Sr. 


State  Spon... 


National  D.. 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


5-17 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


mMDA 


Table  5-15  3D  Multi-Touch  Table 


GIS  Awareness:  3D  Touch  Table  Demo  at  ESRI  Co... 


This  3D  Touch  Table,  demonstrated  at  the  2005  ESRI  conference,  supports  multiple  simultaneous  users 
and  has  an  internal  mechanism  that  enables  it  to  deform  into  software-defined  3d  structures.  The  table  - 
top  is  covered  in  silicone,  and  when  the  table  is  told  that  it  shall  project  heights,  for  example  from 
Google  Earth,  it  raises  small  plastic  sticks  from  underneath  the  elastic  silicone  skin,  and  thereby  'real 
mountains'  can  rise  from  the  table,  (explanation  is  from  http://velvetgrass.blogspot.com/2007/Q4/3d- 
terrain-touch-table-screen.html).  The  scene  is  projected  onto  the  surface  from  above. 


As  the  ocean  has  no  elevation,  it  is  not  clear  that  this  would  be  a  significant  advantage  for  the  Navy. 


Visual  Soph:  Y/N 

Interactive:  Y 

Exploratory:  Y 

Analytical:  N 

Collaborative:  Y 

Potential  Maritime  Application:  VOI  Dossier  and  Track,  What  is  Normal  Behaviour,  Wide  Area 

Surveillance,  Surveillance  Coverage,  Special  Event 

Vendor:  San  Diego  State  University, 

Video: 

References:  [115] 
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Table  5-16  Assessment  Wall 


The  Assessment  Wall  is  a  large-format  wall  display  and  software  designed  to  help  users  more  easily 
monitor  and  explore  continually  updating  text  sources  such  as  news  or  reports,  through  a  high-resolution 
touch-screen  interface.  The  goal  is  to  help  users  find  the  latest  on  topics  known  to  be  of  interest  along 
with  emergent  and  unexpected  themes.  Additional  interactivity  allows  discovery  of  relationships  and 
patterns,  all  in  context  of  the  broader  streaming  data.  The  current  version  of  the  Assessment  Wall  is 
available  as  part  of  IN-SPIRE. 


Visual  Soph:  ? 

Interactive:  ? 

Exploratory: 

7 

Analytical:  ? 

Collaborative:  Y 

Potential  Maritime  Application:  Hardware:  VOI  Dossier  and  Track,  Wide  Area  Surveillance, 

Surveillance  Coverage,  Special  Event.  With  IN-SPIRE:  see  Table  5-8. 

Vendor:  Pacific  Northwest  National  Lab 

Video: 

References:  [99] 
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A  ACRONYMS 


3D 

ACE 

ACM 

AI 

AIS 

ARMAD 

AVIZ 

CAD 

CCI 

CERATOPS 

CF 

CKID 

CMV 

CNVAC 

COEs 

Cdr 

CORA 

DB 

DHS 

DND 

DRDC 

DyDAn 


Three  Dimensional 

Attribute  Correction  Engine 

Association  for  Computing  Machinery 

Artificial  Intelligence 

Automatic  Identification  System 

Automated  Reasoning  for  Maritime  Anomaly  Detection 

Analysis  and  Visualization 

Computer  Aided  Design 

Command,  Control,  &  Interoperability 

Center  for  the  Extraction  and  Summarization  of  Events  and  Opinions  in  Text 
Canadian  Forces 

Center  for  Knowledge  Integration  and  Discovery 

Coordinated  and  Multiple  Views 

Canadian  National  Visualization  and  Analytics  Center 

Centers  of  Excellence 

Commander 

Centre  for  Operational  Research  and  Analysis 
Database 

Department  of  Homeland  Security 
Department  of  National  Defence 
Defence  Research  and  Development  Canada 
Center  for  Dynamic  Data  Analysis 
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ENVI 

ESRI 

FODAVA 

GAV 

GI  Science 

HSIP 

IAIS 

IDS 

IEEE 

IGD 

IJVAAIM 

ILOG 

INRIA 

LIDAR 

MDACIV 

MIAS 

MSPA 

MUSIC 

NDV 

NEVAC 

NSF 

NVAC 

PARVAC 

PNNL 

PURVAC 

RAT 

RJOC 

RMP 

SCIENCE 

SFU 

SIGKDD 


Environment  for  Visualizing  Imagery 

Environmental  Systems  Research  Institute 

Foundations  on  Data  Analysis  and  Visual  Analytics 

Geo  Analytics  Visualization 

Geographic  Information  Science 

Homeland  Security  Infrastructure  Program 

Institut  fur  Intelligente  Analyse-  und  Informationssysteme 

Institute  for  Discrete  Sciences 

Institute  of  Electrical  and  Electronic  Engineers 

Institute  for  Computer  Graphics  Research  in  Darmstadt 

International  Journal  of  Visual  Analytics  for  Advanced  Information  Management 
An  IBM  Company  in  Gentilly,  France  (no  published  acronym  explanation) 
Institut  national  de  recherche  en  informatique  et  automatique 
Light  Detection  and  Ranging 

Maritime  Domain  Analysis  through  Collaboration  and  Interactive  Visualization 
Multimodal  Information  Access  and  Synthesis 
Maritime  Security  Patrol  Area 

Multi-Sensor  Integration  within  a  Common  Operating  Environment 

N-Dimensional  Visualizer 

North-East  Visualization  and  Analytics  Center 

US  National  Science  Foundation 

National  Visualization  and  Analytics  Center 

Pacific  Rim  Visualization  and  Analytics  Center 

Pacific  Northwest  National  Laboratory 

Purdue  University  Regional  Visualization  and  Analytics  Center 
RMP  Analysis  Toolset 
Regional  Joint  Operations  Centre 
Recognized  Maritime  Picture 

Spatial  Cognition  and  Interactive  Expertise  in  Natural  and  Computational 
Environments 

Simon  Fraser  University 

Special  Interest  Group  on  Knowledge  Discovery  and  Data  Mining 


A-2 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


MDA 


Ref:  RX-RP-52-8724 

Issue/Revision:  1/0 

Date:  OCT.  28,  2009 


SPIE 

SRVAC 

TVCG 

UBC 

UNITAR 

UNOSAT 

US 

VA 

VAC 

VACCINE 

VAKD 

VAMA 

VAR 

VAST 

VISAD 

VOI 

VR 


Society  of  Photo-Optical  Instrumentation  Engineers 

Southeastern  Regional  Visualization  and  Analytics  Center 

Transactions  on  Visualization  and  Computer  Graphics 

University  of  British  Columbia 

UN  Institute  for  Training  and  Research 

UNITAR  Operational  Satellite  Applications  Programme 

United  States 

Visual  Analytics 

Visual  Analytics  Consortium 

Visual  Analytics  for  Command,  Control,  and  INteroperability  Environments 

Workshop  on  Visual  Analytics  and  Knowledge  Discovery 

Visual  Analytics  for  Maritime  Awareness 

Visual  Analytics  Research  Lab  at  SFU 

Visual  Analytics  Science  and  Technology 

Visualization  for  Algorithm  Development 

Vessel  of  Interest 

Virtual  Reality 
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B  ANNOTATED  BIBLIOGRAPHY 


[1]  ACM  Special  Interest  Group  on  Knowledge  Discovery  and  Data  Mining  (SIGKDD),  SIGKDD 
Explorations  Special  Issue:  Visual  Analytics, 

http://www.sigkdd.org/explorations/issue.php?volume=9&issue=2&vear=2007&month=12.,  2007. 

[See  file:  ACM  SIGKDD  Explorations  Special  Issue.mht] 

This  is  an  example  of  a  Non-VA  Journal  publishing  a  Visual  Analytics  special  issue. 

[2]  W.  Aigner,  "Visualization  of  time-oriented  data  visualization  techniques," 
http://ieg.ifs.tuwien.ac.at/~aigner/presentations/TimeViz  techniques0809  lup.pdf,  2009. 

[A  copy  is  attached  as  file:  AignerTimeViz.pdf] 

Very  interesting  and  useful  ideas  on  how  to  visualize  time-varying  data. 

[3]  G.  Andrienko,  and  N.  Andrienko,  “Exploration  of  Massive  Movement  Data:  a  Visual  Analytics 
Approach,”  in  AGILE,  Girona  Spain,  2008. 

[A  copy  is  attached  as  file:  AndrienkoAgile08.pdf] 

[4]  G.  Andrienko,  and  N.  Andrienko,  “Spatio-temporal  aggregation  for  visual  analysis  of  movements,”  in 
Visual  Analytics  Science  and  Technology,  2008.  VAST  '08.  IEEE  Symposium  on,  2008,  pp.  51-58. 

An  important  paper  on  ways  to  visualize  and  explore  the  statistical  patterns  in  vessel  motion.  Are 
there,  for  example,  whole  clusters  of  vessel  that  travel  up  the  St.  Lawrence  Seaway? 

[5]  G.  Andrienko,  N.  Andrienko,  P.  Jankowski  et  al.,  “Geovisual  analytics  for  spatial  decision  support: 
Setting  the  research  agenda,”  International  Journal  of  Geographical  Information  Science,  vol.  21,  no. 
8,  pp.  839-857,  September  2007,  http://infovis.uni- 

konstanz.de/papers/2007/VisualGeoAnalytics  VASDS-IJGIS2007.pdf,  2007. 

[A  copy  is  attached  as  file:  Andrienko_GeoVisualAnalytics.pdf] 

This  agenda-setting  paper  makes  the  argument  that  GeoVisual  Analytics  should  be  a  recognized 
subtype  of  visual  analytics.  It  highlights  the  following  challenges  of  geospatial  visual  analytics: 
complex  nature  of  geographic  space,  multiple  actors  with  different  roles  (hence  a  need  for 
collaboration,  communication,  and  flexibility),  and  tacit  criteria  and  knowledge. 
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[6]  G.  Andrienko,  N.  Andrienko,  and  S.  Wrobel,  “Visual  Analytics  Tools  for  Analysis  of  Movement 
Data,”  in  ACM  SIGKDD  Explorations  http://geoanalytics.net/and/papers/sigkdd07.pdf,  2007,  pp.  38- 
46.  [A  copy  is  attached  as  file:  AndrienkoSigkdd07.pdf] 

A  useful  paper  exploring  ways  to  visualize  repeated  motion  patterns,  using  multiple  linked  displays. 
Introduces  visualizations  that  key  on  daily  and  weekly  patterns,  but  also  allow  you  to  extract  and 
view  a  single  trip,  linked  to  a  sliding  time  filter. 

[7]  ACM  SIGKDD  Workshop  on  Visual  Analytics  and  Knowledge  Discovery, 
http://portal.acm.org/toc.cfm7idM  562849&tvpe=proceeding&coll=Portal&dl=GUIDE&CFID=54542 

277&CFTQKEN-77900280.  2009. 

[See  file:  VAKD09.mht] 

[8]  K.  D.  Bailey,  Typologies  and  Taxonomies:  An  Introduction  to  Classification  Techniques  p.App.  96: 
Sage  Publications,  Inc,  1994. 

A  useful  monograph  about  classification  schemes. 

[9]  P.  Bak,  F.  Mansmann,  H.  Janetzko  et  al.,  “Spatiotemporal  Analysis  of  Sensor  Logs  using  Growth 
Ring  Maps,”  in  Visual  Analytics  Science  and  Technology,  Atlantic  City,  New  Jersey,  IEEE, 
http://infovis.uni-konstanz.de/papers/2009/bak.pdf,  2009. 

[A  copy  is  attached  as  file:  bakGrowthRings.pdf] 

This  paper  is  marked  "visual  analytics"  but  is  really  just  a  visualization  strategy >.  It  shows  a  way  to 
visually  summarize  long-term  time  patterns  of  activity  using  "growth  rings. " 

[10]  V.  Barash.  "Excel  .Net  Map,"  web  page  at: 
http://www.socialtext.net/ocu2008/index.cgi7excel  net  map  sna  toolkit,  2008. 

Relationships  are  edges  in  a  directed  map.  Nodes  are  individuals  and  are  shown  as  tagged  semi¬ 
transparent  bubbles,  with  size  indicating  some  parameter.  The  thing  of  interest  is  the  creative 
spacing  of  the  bubbles  into  a  sort  of  ring  galaxy.  This  is  not  Visual  Analytics,  rather  visualization, 
but  the  portrayal  might  be  useful  in  a  VA  solution. 

[11]  B.  Beaulieu,  “Homeland  Security  through  the  Palanterra,”  Geointelligence,  pp.  14-19,  July/August 
2004,  http://www.nima.mil/NGASiteContent/StaticFiles/OCR/GeoQ704.pdf.  2004. 

[A  copy  is  attached  as  file:  Palenterra.pdf] 

[12]  E.  Bertini,  and  D.  Lalanne,  “Surveying  the  complementary  role  of  automatic  data  analysis  and 
visualization  in  knowledge  discovery,”  in  VAKD'09,  Paris,  http://www.hiit.fi/vakd09/vakd09bertini- 
cr.pdf,  2009.  [A  copy  is  attached  as  file:  Bertini_VAKD_09.pdf] 

Makes  the  argument  that  data  visualization  and  data  mining  need  to  be  better  integrated. 

[13]  M.  Bloch,  and  S.  Carter.  "Twitter  Chatter  During  the  Super  Bowl,"  web  page  at: 
http://www.nvtimes.com/interactive/2009/02/02/sports/200902Q2  superbowl  twitter.html,  2009.  [A 
copy  is  attached  as  file:  SuperBowlTweets.png  SuperbowlTwitter.avi] 

Uses  word  clouds  to  show  what  people  were  tweeting  during  the  Superbowl.  The  data  can  be 
visualized  in  three  dimensions:  geograhically  (words  are  superimposed  on  a  map  of  the  USA), 
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[14] 


[15] 


[16] 


[17] 


[18] 


[19] 


temporally  (the  viewer  moves  back  and  forth  in  time  using  a  slider  across  the  top  of  the  page,  and 
word  clouds  change  instantly),  and  by  subject  (by  clicking  on  a  menu  down  the  top  left  side).  Users 
can  drill-down  to  see  exact  word-counts  by  mousing-over  a  city  or  region. 

E.  K.  Bowman,  D.  Gouin,  P.  Houghton  et  al.,  "Intense  Collaboration:  Knowledge  Integration  Task 
(KnowIT)  Results,"  TR-C3I-TP2- 1-2008,  TTCP  C3I  Group;  Technical  Panel  2  2008. 

[A  copy  is  attached  as  file:  Intense  Collaboration  KnowIT  report  Aug  14  (2). doc] 

Lays  out  the  management  methodologies  and  team  interaction  standards  to  be  used  when  scientific 
teams  from  different  agencies  and  different  allied  countries  have  to  work  together  intensely. 

S.  E.  Brennan,  K.  Mueller,  G.  Zelinsky  et  al.,  “Toward  a  Multi-Analyst,  Collaborative  Framework  for 
Visual  Analytics,”  in  IEEE  Symposium  On  Visual  Analytics  Science  And  Technology  (VAST  06) 
IEEE,  http://www.psvchology.sunysb.edu/sbrennan-/papers/VAST.pdf,  2006. 

[A  copy  is  attached  as  file:  Brennan_CollabFramework.pdf] 

The  paper  sketches  a  method  by  which  multiple  analysts  build  and  maintain  separate  analytical 
models  and  visualizations,  but  may  at  any  time  super-impose  a  connected  view’  of  the  analytical 
models  of  another  analyst.  Views  are  node-maps,  where  common-ground  might  be  for  example 
common  data  points  used. 

B.  Brewin,  “Intelligence  Community  Developing  a  Virtual  World  for  Analytics,”  Government 
Executive  .com,  Nove  7  2007,  http://www.virtualworldsnews.eom/2007/l  1/intelligence- 
co.html#more,  2007. 

[A  copy  is  attached  as  file:  A-SpaceX.mht] 

This  is  a  brief  article  mentioning  a  US  project  Analyst  Space  for  Exploitation  (A-SpaceX)  aimed  at 
exploiting  virtual  worlds  for  information  analysis.  Jeffery >  Morrison,  who  runs  the  program,  wants  to 
build  in  visual  metaphors  for  information  that  he  calls  "mind  snaps. "  The  article  suggests  "virtual 
worlds  could  be  golden  for  the  future  of predictive  analysis. " 

B.  Brown,  “Intelligence  community  developing  virtual  world  analysis  tools,”  Government  Executive, 
November  7,  2007  http://www.govexec.eom/dailyfed/l  107/1 10707bbl.htm,  2007. 

[A  copy  is  attached  as  file:  Brown  A-SpaceX.mht] 

A  brief  magazine  article  about  A-SpaceX,  introducing  the  plan  to  create  virtual  worlds  where 
analysts  can  explore  intelligence  questions. 

"Hierarchical  multi-touch  selection  techniques  for  collaborative  geospatial  analysis," 
http://link.aip.Org/link/7PSI/7346/73460D/l,  http://dx.doi.Org/10.l  1 17/12.819025,  2009. 

[See  file:  ButkiewiczMultiTouch.pdf] 

The  multi-touch  table  was  found  to  be  "powerful  but  non-intimidating;  a  system  that  allows  and 
encourages  complex  combinations  of  actions,  but  can  be  operated  without  any  training. " 

S.  Butner,  W.  Cowley,  V.  Crow  et  al.,  “A  Look  at  First  Look,”  VAC  Views,  pp.  14-15,  November 
2008,  2008.  [A  copy  is  attached  as  file:  VACViews05_nov08.pdf] 

First  Look  monitors  information  streams  and  comments  to  the  operators  when  elements  of  the 
information  streams  change. 
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[20]  C.  E.  Carlsen.  "Metasyn  Interactive  information  visualization,"  web  page  at: 
http://sixthsensor.dk/work/6,  2008.  [A  copy  is  attached  as  file:  Larsen_Metasyn] 

This  is  a  good  technical  summary  of  how  Carlsen  conceptualized  Metasyn,  using  three  levels:  Meta, 
Mesa,  and  Micro.  It  also  includes  three  screen-shots  that  are  helpful.  The  installation  was  on  a  very 
large  curved  screen. 

[21]  C.  E.  Carlsen,  and  SixthsensorStream,  "Metasyn,"  Youtube  Video  available  on-line  at: 
http://www.youtube.com/watch7vMlXJuJnEElclY,  2009. 

[A  copy  is  attached  as  file:  metasyn_video.mp4] 

An  interesting  "3D  World”  visualization  of  a  corpus  of  art.  Interesting  features  include:  lines  that 
show  links  between  related  objects,  but  that  fade  into  the  distance,  navigation  abilty  so  that  lines  fade 
or  defade,  leaving  the  forground  clear,  different  types  of  links  are  different  colours,  you  can  drill 
down  on  each  node. 

[22]  CERATOPS.  "Center  for  the  Extraction  and  Summarization  of  Events  and  Opinions  in  Text  "  web 
page  at:  http://www.cs.pitt.edu/mpqa/ceratops/.  2009. 

[A  copy  is  attached  as  file:  CERATOPS  Home  Page.mht] 

CERATOPS  looks  for  subtle  information  such  as  opinions  and  beliefs  in  free  text. 

[23]  S.-M.  Chan,  L.  Xiao,  J.  Gerth  et  al.,  “Maintaining  interactivity  while  exploring  massive  time  series,” 
in  Visual  Analytics  Science  and  Technology,  2008.  VAST  '08.  IEEE  Symposium  on,  2008,  pp.  59- 
66. 

Because  visual  analytics  required  real-time  responsiveness  from  the  computer,  systems  accessing 
huge  amounts  of  data  must  be  implemented  innovatively.  This  is  one  example  of  such  an  architecture. 

[24]  C.  Chen,  “An  Information-Theoretic  View  of  Visual  Analytics,”  Computer  Graphics  and 
Applications,  IEEE,  vol.  28,  no.  1,  pp.  18-23,  2008. 

[A  copy  is  attached  as  file:  Chen06.pdf] 

Provides  theoretical  guidelines  for  designing  visual  analytic  systems  so  that  the  visualizations  are  not 
deceptive. 

[25]  Y.  Chen,  J.  Yang,  and  W.  Ribarsky,  “Toward  effective  insight  management  in  visual  analytics 
systems,”  in  Visualization  Symposium,  2009.  PacificVis  '09.  IEEE  Pacific,  2009,  pp.  49-56. 

The  paper  lays  out  a  knowledge  management  methodology >  that  should  (it  suggests)  work  in  parallel 
to  the  visualization,  as  part  of  the  visual  analytics. 

[26]  CNVAC.  "Canadian  Network  of  Visual  Analytics  Centres,"  web  page  at:  http://www.cnvac.ca/,  2008. 
[A  copy  is  attached  as  file:  CnvacHomePage.mht] 

A  rather  thin  front-page  for  the  Canadian  network  of  visual  analytics  centres. 

[27]  M.  Davenport,  "Maritime  Domain  Awareness  Knowledge  Management  Requirements,"  CR  2007- 
174,  DRDC-Valcartier,  2007. 

An  analysis  of potential  requirements  for  knowledge  management  capabilities,  at  the  RJOCs. 
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[28]  M.  Davenport,  "MISR-Vis  Maritime  Intelligence,  Surveillance,  and  Reconnaissance  Knowledge 
Base,"  CR  2007-543,  DRDC  Valcartier,  2007. 

Describes  the  development  of  an  ontology >  of  visualization  strategies  and  their  potential  application  to 
maritime  domain  awareness. 

[29]  M.  Davenport,  "Maritime  Anomaly  Detection  Workshop  Report  and  Analysis,"  CR  2008-275, 
DRDC-Valcartier,  2008. 

Reports  on  a  workshop  held  in  Valcartier  in  2008  with  subject-matter  experts  from  RJOC(A),  in 
which  a  set  of  maritime  anomalies  was  brainstormed  and  refined.  Provides  a  taxonomy  of  37 
anomalies. 

[30]  M.  Davenport,  Opportunities  for  Applying  Visual  Analytics  for  Maritime  Awareness,  2009. 

[31]  M.  Davenport,  "Opportunities  for  Applying  Visual  Analytics  for  Maritime  Awareness,"  CR  2009- 
227,  DRDC-Valcartier,  2009. 

This  is  the  second  of  two  reports  under  the  Visual  Analytics  for  Maritime  Awareness  contract. 

[32]  M.  Davenport,  and  C.  Risley,  "Information  Visualization:  the  State  of  the  Art  for  Maritime  Domain 
Awareness,"  CR  2006-122,  DRDC  Atlantic,  http://pubs.drdc-rddc.gc.ca,  2006. 

An  extensive  survey  of  research  and  commercial  software  and  hardware  that  is  of  relevance  to 
visualization  of  the  maritime  domain. 

[33]  M.  Davenport,  E.  Widdis,  and  J.  Rafuse,  "RMP  Baseline  Update,  Vols  1-3,"  TR  2005-293,  TR  2005- 
294,  TR  2005-295,  DRDC-Atlantic,  2005. 

An  inventory  of  surveillance  assets  and  surveillance  data  processing  methods,  done  as  a  foundation 
for  the  MUSIC  project. 

[34]  Third  Annual  DHS  University  Network  Summit,  http://www.orau.gov/DHSsummit/program.pdf, 

2009.  [See file:  DhsUniversityNetworkSummit2009.pdf] 

This  appears  to  be  a  team-building  workshop  rather  than  an  academic  conference.  There  are 
apparently  no  proceedings  published. 

[35]  DHS  Science  and  Technology  Directorate,  "Five-Year  Research  and  Development  Plan;  Fiscal  Years 
2008-2013,"  H.  Security,  ed.,  Department  of  Homeland  Security, 
http://www.dhs.gov/xlibrary/assets/SandT5vearplan.pdf,  2008. 

[A  copy  is  attached  as  file:  DhsSandT5yearplan.pdf] 

Shows  the  US  Department  of  Homeland  Security  Science  and  Technology >  goals  and  budget  for  the 
next  five  years  (about  US$4. 5M  per  year  for  basic  VA  research,  plus  about  US$3M  per  year  for 
university  centers  of  excellence  in  Command,  Control,  and  interoperability  (CCI)) 

[36]  DyDAn.  "The  Homeland  Security  Center  for  Dynamic  Data  Analysis  "  web  page  at: 
http://dvdan.rutgers.edu/,  2009. 

Announcement  of  the  creation  of  DyDAn  at  Rutgers,  with  an  overview  of  its  mission. 
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[37]  S.  Eaton,  “Visualization  and  Analytics  Centers,”  VAC  Views,  pp.  20-27,  Nov  2008,  2008.  [A 

copy  is  attached  as  file:  VACViews05_nov08.pdf] 

A  useful  list  of  all  the  VAC  consortium  members  as  of 2008,  with  brief  notes  on  what  research  they 
are  doing. 

[38]  J.  A.  Fails,  A.  Karlson,  L.  Shahamat  et  al.,  “A  Visual  Interface  for  Multivariate  Temporal  Data: 
Finding  Patterns  of  Events  across  Multiple  Histories,”  in  Proceedings  of  IEEE  Symposium  on  Visual 
Analytics  Science  and  Technology  (VAST  2006),  IEEE, 

http://research.microsoft.com/pubs/75850/fails-VAST2006.pdf,  2006,  pp.  167-174. 

[A  copy  is  attached  as  file:  Fails_PattemFinder.pdf] 

Provides  one  way  to  overlay  extended-time  events,  in  this  case  medical  drug  interactions  as  a 
function  of period  of  medication.  Much  of  their  discussion  is  looking  at  how  to  display  tabular  data 
(text)  in  small  tables  associated  with  events.  Events  are  plotted  on  a  horizontal  time-line. 

[39]  G.  Falkman,  T.  Ziemke,  and  M.  Riveiro.  "Information  visualization  for  ground  situation  awareness," 
web  page  at:  http://www.his.se/english/research/infofusion/research/scenarios-and-proiects/gsa/gsa2- 
visualization/,  2009.  [A  copy  is  attached  as  file:  VisadPage.mht] 

Shows  the  architecture  ofVISAD:  an  interactive  and  visual  analytical  tool  for  the  detection  of 
behavioral  anomalies  in  maritime  traffic  data.  This  includes:  including  the  user  in  the  fusion 
process,  visualization  of  large  data  sets,  visualization  of  uncertainty,  and  different  levels  of 
granularity  in  time  and  space. 

[40]  S.  Few,  "Visualizing  Change:  An  Innovation  in  Time-Series  Analysis,"  SAS,  www.jmp.com/few, 
2007.  [A  copy  is  attached  as  file:  Few_visualizing_change.pdf] 

[41]  B.  Fisher,  "VACCINE:  Visual  Analytics  for  Command,  Control,  and  INteroperability  Environments," 
Simon  Fraser  University,  2009.  [A  copy  is  attached  as  file:  SFU_VACCINE.pdf] 

Our  best  description  (so  far)  of  the  new  VACCINE  organization  for  visual  analytics. 

[42]  D.  Fisher,  A.  Hoff,  G.  Robertson  et  al.,  “Narratives:  A  visualization  to  track  narrative  events  as  they 
develop,”  in  Visual  Analytics  Science  and  Technology,  2008.  VAST  '08.  IEEE  Symposium  on, 
http://research.microsoft.com/apps/pubs/default.aspx?id=78917,  2008,  pp.  115-122. 

[A  copy  is  attached  as  file:  FisherNarratives.pdf] 

A  text-analysis  tool  that  looks  for  themes  and  associated  keywrds,  and  then  tracks  how  those 
keywords  evolve  over  time. 

[43]  FODAVA.  "NSF/DHS  FODAVA-Lead:  Foundations  of  Data  and  Visual  Analytics,"  web  page  at: 
http://  fodava.  gatech.edu/,  2009.  [A  copy  is  attached  as  file:  Welcome  to  FODAVA.mht] 

FODAVA  is  the  newly-set-up  organization  based  at  Georgia  Tech,  operating  with  funding  from  the 
National  Science  Foundation  and  the  Department  of  Homeland  Security 
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[44]  Future  Point  Systems  Inc.,  "Starlight  Demo  -  Text  and  UAV  Video  Analysis,"  Youtube  available  on¬ 
line  at:  http://www.voutube.com/watch?v=dUtvlb8cPqo&NR=l,  2008. 

[A  copy  is  attached  as  file:  YouTube  -  Starlight  and  UAVs.mht] 

This  video  is  quite  recent ,  and  walks  through  a  scenario  in  Afghanistan,  assembling  new  reports  of 
rocket  attacks  on  airports  and  cross-referencing  to  available  UA  V  flights.  Useful  for  seeing  the 
CONOPS  of  Starlight  and  the  interactions  with  the  software. 

[45]  Future  Point  Systems  Inc.  "Products:  Starlight,"  web  page  at: 
http://wwvv.fliturcpointsvstcms.com/7pagc~products.  2009. 

[A  copy  is  attached  as  file:  StarlightHomepage.mht] 

Starlight  is  an  early  and  still  leading  visual  analytics  software  suite.  It  can  portray  network 
diagrams,  free-topic  clusters  of  words,  geospatial  relationships,  temporal  relationships,  and 
hierarchical  models  in  an  abstract  analysis  space,  with  links  to  maps,  photos,  text  documents,  etc. 

[46]  General  Dynamics,  "Command  Post  of  the  Future,"  G.  Dynamics,  ed., 
http://www.gdc4s.com/documents/cnof-9-12-06.ndf.  2006. 

[A  copy  is  attached  as  file:  Cpof_Brochure.pdf] 

This  is  the  quick  introduction  to  the  features  of  Command  Post  of  the  Future,  which  is  mentioned  in 
Illuminating  the  Path. 

[47]  Georgia  Tech.  "Sample  Detailed  Answer  (extracted  from  the  VAST  2007  contest  entry  submitted  by 
Georgia  Tech),"  web  page  at:  http://www.cs.umd.edu/hcil/VASTchallenge08/ 
ExamplefromVAST2007contest-GT-Jigsaw/index.htm,  2007. 

[A  copy  is  attached  as  file:  JigsawGeorgiaTech.mht] 

This  summary  of  the  winning  Jigsaw  entry  into  the  VASt  2007  "Blue  Iguanadon  "  contest  semes  as  a 
good  introduction  to  the  Jigsaw  tool,  complete  with  some  frank  comments  on  what  capabilities  they 
need  but  don 't  yet  have. 

[48]  Georgia  Tech.  "Visual  Analytics  Digital  Library,"  web  page  at:  http://vadl.cc.gatech.edu/,  2009.  [A 
copy  is  attached  as  file:  Visual  Analytics  Digital  Library.mht] 

This  is  mainly  a  collection  of  lecture  notes  and  homework  assignments  from  the  Georgia  Tech  center 
of  excellence. 

[49]  M.  Ghoniem,  D.  Luo,  J.  Yang  et  a!.,  "NewsLab:  Exploratory  Broadcast  News  Video  Analysis,"  CVC- 
UNCC-07-1 1,  UNC  Charlotte,  http://www.viscenter.uncc.edu/TechnicalReports/CVC-UNCC-07- 

11. pdf.  2007. 

[A  copy  is  attached  as  file:  NewsLab.pdf] 

Uses  a  river  metaphor  is  used  to  depict  the  thematic  changes  of  the  news  over  time,  together  with  a 
lens  for  drilling-down  for  details.  Data  is  extracted  from  the  closed  captions  on  the  TV  new  casts. 

Can  be  used  to  find  emerging  topics  in  the  news,  compare  different  broadcasters,  or  mine  the  news 
for  topics  of  interest. 
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[50] 


[51] 


[52] 


[53] 


[54] 


[55] 


I.  Global,  International  Journal  of  Visual  Analytics  for  Advanced  Information  Management 
(IJVAAIM),  http://www.igi-global.com/ioumals/details.asp7icN8084.,  2010. 

[ See  file:  IJVAAIM.mht] 

This  journal  is  just  starting  up,  but  could  be  important  as  the  first  journal  (VAC  Views  is  a  magazine, 
not  a  journal)  dedicated  to  visual  analytics. 

A.  Godwin,  R.  Chang,  R.  Kosara  et  al.,  “Visual  analysis  of  entity  relationships  in  the  Global 
Terrorism  Database,”  SPIE,  http://link.aip.Org/link/7PSI/6983/69830G/l 
http://dx.doi.Org/10.l  1 17/12,778084.  2008,  pp.  69830G. 

Uses  algorithms  from  gene  sequencing  to  find  matches  between  patterns  of  activity  of  terrorist 
groups,  thus  identifying  probable  collaborations,  and  possibly  attributing  previously  unattributed 
actions. 

M.  C.  Hao,  U.  Dayal,  and  D.  A.  Keim,  “Visual  analytics  techniques  for  large  multi-attribute  time 
series  data,”  in  Visualization  and  Data  Analysis  2008,  SPIE,  http://infovis.uni- 
konstanz.de/papers/2008//EI  1 08-EI 1 07-32Submit.pdf,  2008,  pp.  680908. 

[A  copy  is  attached  as  file:  HaoTimeSeries.pdf] 

Transforms  a  very  large  multi-attribute  time  series  into  a  series  of  coloured  icons,  with  the  shape  of 
the  icons  encoding  the  multiple  attributes  so  that  changes  over  time  can  be  seen  visually. 

M.  C.  Hao,  U.  Dayal,  D.  A.  Keim  et  al.,  “Visual  analytics  of  anomaly  detection  in  large  data 
streams,”  SPIE,  http://link.aip.Org/link/7PSI/7243/72430B/l.  http://dx.doi.Org/10.l  1 17/12.810945, 
2009,  pp.  72430B. 

This  is  mainly  a  presentation  of  ways  to  detect  and  display  anomalies  in  massive  data  streams.  Not 
much  of  an  anaytical  tool. 

Q.  Ho,  and  M.  Jem,  “Exploratory  3D  Geovisual  Analytics,”  in  Research,  Innovation  and  Vision  for 
the  Future,  2008.  RIVF  2008.  IEEE  International  Conference  on,  2008. 

Describes  the  GeoAnalytics  Visualization  component  toolkit,  using  many  example  from  maritime 
surveillance.  The  toolkit  is  able  to  represent  patterns  in  geography  amd  time,  as  well  as  attribute 
spaces. 

J.  Hollan,  E.  Hutchins,  and  D.  Kirsh,  “Distributed  cognition:  toward  a  new  foundation  for  human- 
computer  interaction  research,”  ACM  Trans.  Comput.-Hum.  Interact.,  vol.  7,  no.  2,  pp.  174-196, 
http://portal.acm. org/citation.cfm?doid=353485.353487#,  2000. 

[A  copy  is  attached  as  file:  hollanDistributedCog.pdf] 

A  seminal  paper  on  the  cognitive  science  behind  visual  analytics,  and  on  new  paradigms  for  thinking 
of  collaboration. 
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[56] 

[57] 

[58] 

[59] 

[60] 

[61] 

[62] 

[63] 


Homeland  Security,  "Homeland  Security  Presidential  Directives,"  H.  Security,  ed., 
http://www.dhs.gov/xabout/laws/editorial  0607.shtm,  2009. 

[A  copy  is  attached  as  file:  DHS  Homeland  Security  Presidential  Directives. mht] 

A  list  of  24  presidential  directives  that  sen’e  as  a  mandate  for  DHS.  Directives  3,  5,  7,  8,  9,  10,  21, 

23  are  directly  relevant  to  visual  analytics. 

E.  Hovy,  P.  Pantel,  and  D.  McLeod,  "CKID:  The  Center  for  Large-Scale  Strategic  Knowledge 
Integration  and  Discovery  from  Multiple  Media,"  Information  Sciences  Institute, 
http://www.orau.gov/DHSSummit/2007/Presentations/Hovy.pdf  2007. 

[A  copy  is  attached  as  file:  HovyCkid.pdf] 

Provides  an  overview  of  the  CKID  objectives  and  activities. 

12.  "i2  Analyst's  Notebook:  Assisted  Analysis  and  Visualization,"  web  page  at: 
http://www.i2inc.com/products/analysts  notebook/,  2009. 

[A  copy  is  attached  as  file:  12  Analysts  Notebook.mht] 

12  is  well-established  and  popular  among  defence  agencies.  It  provides  an  interactive  visual 
environment  in  which  to  build  hypotheses  and  view  patterns,  bringing  information  from  linked 
databases. 

F.  IAIS.  "Visual  Analytics,"  web  page  at:  http://www.iais.fraunhofer.de/4801.html?&L=l.  2009.  [A 
copy  is  attached  as  file:  Fraunhofer  IAIS. mht] 

This  group  (under  Gennady  Andrienko)  focuses  in  temporal  and  geospatial  data,  such  as  trajectories 
of  moving  objects,  time  series  data,  and  events. 

Fifth  International  Conference  on  Coordinated  &  Multiple  Views  in  Exploratory  Visualization 
http://ieeexplore.ieee.org/xpl/tocresult. jsp?isnumber=4269933&isYear=2007,  2007. 

Years  before  the  term  "visual  analytics"  was  coined,  academics  were  meeting  at  this  conference  to 
discuss  this  central  technology >  of  Visual  Analytics.. 

IEEE  Symposium  on  Visual  Analytics  Science  and  Technology >  2007 
http://conferences.computer.org/vast/vast2007/.  2007. 

[See  file:  VAST  2007.mht] 

IEEE  Symposium  on  Visual  Analytics  Science  and  Technology  08 
http://vis.computer.org/VisWeek2008/VAST/index.html,  2008. 

This  is  the  leading  annual  conference  on  Visual  Analytics. 

EuroVis  2009  Workshop  Data  Management  &  Visual  Analytics  ’  http://www.vismaster.eu/workshop- 
berlin.  2009.  [See file:  EuroVis  2009  Workshop.mht] 

This  one-day  workshop  was  initiated  by  VisMaster,  to  coincide  with  EuroVis  2009.  The  10  technical 
papers  are  not  yet  available  on-line,  but  the  titles  suggest  that  they  are  not  directly  relevant  to 
Maritime  Domain  Awareness. 
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[64]  IEEE,  IEEE  Transactions  on  Visualization  and  Computer  Graphics  ( TVCG ):  IEEE, 
http://www2.computer.org/portal/web/tvcg/,  2009. 

[65]  F.  IGD.  "Realtime  Solutions  for  Simulation  and  Visual  Analytics,"  web  page  at: 
http://www.igd.fraunhofer.de/igd-a3/index.html,  2009. 

A  core  research  topic  at  Fraunhofer  is  Visual  Analytics  for  interactive  visualization  and  intelligent 
complexity  reduction ,  which  is  understood  to  be  an  interdisciplinary  combination  of  data  mining  and 
information  visualization. 

[66]  INRIA.  "AVIZ  Project-Team  -  Analysis  and  Visualization  "  web  page  at: 
http://www.inria.fr/saclav/research/research-teams/EqR  AVIZ/view,  2009. 

[A  copy  is  attached  as  file:  AVIZ  —  INRIA  Saclay.mht] 

This  is  the  home  page  for  A  VIZ,  which  is  developing  methods  to  visualize  and  smoothly  navigate 
through  large  datasets  (e.g.  Wikipedia),  including  engineering  tools  for  building  visual  analytics 
systems  that  can  access,  search,  visualize  and  analyze  large  datasets  with  smooth,  interactive 
response. 

[67]  Geospatial  Visual  Analytics  Workshop  08,  http://www.geoanalvtics.net/GeoVisualAnalytics08/. 

2008.  [See file:  Geospatial  VA  2008. mht] 

[68]  International  Maritime  Bureau.  "Live  piracy  Map,"  web  page  at:  http://www.icc- 
ccs.org/index.php?option=com  fa  h  r  i  k  &  v  i  c  w = v  i  s  u  a  I  i  z  a  t  i  o  n  &  c  o  n  t  ro  1 1  c  r= v  i  su  a  1  i  z  a  t  i  o  n .  g  o  o  g  1  c  m  a  p  &  1 1  c 

mid=219.  2009. 

[A  copy  is  attached  as  file:  Live  Piracy  Map. mht] 

A  Google  Maps  application  showing  icons  for  pirate  activity  and  drill-down  to  get  details. 

[69]  ITT  Visual  Information  Solutions,  "Military  and  Intelligence  Operations  with  ENVI  "  Video  available 
on-line  at:  http://www.youtube.eom/user/ITTVIS#play/uploads/5/FdekH5Tnrms,  2009. 

[70]  M.  Jem,  and  J.  Franzen,  “"GeoAnalytics"  -  Exploring  spatio-temporal  and  multivariate  data,”  in 
Information  Visualization,  2006.  IV  2006.  Tenth  International  Conference  on,  2006,  pp.  25-31. 

This  is  visual  analytics,  not  just  visualization.  Jem  shows  how  multiple  linked  windows  can  jointly 
display  spatial  and  temporal  information. 

[71]  M.  Jern,  L.  Thygesen,  and  M.  Brezzi,  “Geovisual  Analytics  Web— Enabled  Tools  for  Dissemination 
of  OECD  Regional  Statistics,”  in  Fifth  International  Conference  on  Coordinated  and  Multiple  Views 
in  Exploratory  Visualization  (CMV  2007)  Zurich,  IEEE,  http://www.oecd.org/dataoecd/58/25/ 

4201 1526.pdf,  2007. 

[A  copy  is  attached  as  file:  JemOecdExplorer.pdf] 

This  (draft)  paper  is  probably  th  emost  up-to-date  description  of  the  GAV  visual  analytics  toolkit. 
GAV  was  used  to  build  the  “OECD  eXplorer”,  a  customized  web-compliant  tool  for  interactively 
analyzing  and  communicating  insights  and  discoveries  about  spatial-temporal  and  multivariate 
Organisation  for  Economic  Co-operation  and  Development  (OECD)  regional  data. 
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[72]  J.  Jones,  R.  Chang,  T.  Butkiewicz  et  al.,  “Visualizing  uncertainty  for  geographical  information  in  the 
global  terrorism  database,”  SPIE,  http://link.aip.Org/link/7PSI/6983/69830E/l, 
http://dx.doi.Org/10.l  1 17/12.777695,  2008,  pp.  69830E. 

[A  copy  is  attached  as  file:  JonesUncertainty.pdf] 

A  serious  attempt  at  representing  uncertainty  in  a  situation  map,  in  this  case  using  "Squarified 
treemaps. "  It's  a  tough  problem  and  this  is  probably  not  the  final  solution. 

[73]  T.  Kapler,  R.  Eccles,  R.  Harper  et  al.,  “Configurable  Spaces:  Temporal  analysis  in  diagrammatic 
contexts,”  in  Visual  Analytics  Science  and  Technology,  2008.  VAST  '08.  IEEE  Symposium  on,  2008, 
pp.  43-50. 

"Configurable  Spaces"  is  a  new  analytical  method  based  on  GeoTime  for  visualizing  patterns  of 
activity  over  time  in  complex  diagrams.  The  document  screen-grabs  do  not  appear  to  "clarify"  what  is 
happening,  and  no  YouTube  has  bene  published,  so  it  is  not  clear  yet  how  successful  "Configurable 
Spaces  "  is. 

[74]  T.  Kapler,  and  W.  Wright,  "GeoTime  Information  Visualization,"  Oculus  Info., 
http://vadl.cc.gatech.edu/documents/55  Wright  KaplerWright  GeoTime  InfoViz  Jrnl  05  send.pdf, 

2005. 

[A  copy  is  attached  as  file:  Wright_GeoTime_InfoViz_Jml.pdf] 

[75]  P.  E.  Keel,  “Collaborative  Visual  Analytics:  Inferring  from  the  Spatial  Organization  and 
Collaborative  Use  of  Information,”  in  Visual  Analytics  Science  And  Technology,  2006  IEEE 
Symposium  On,  2006,  pp.  137-144. 

This  is  a  comprehensive  and  information-rich  report  on  a  collaboration  architecture  "EWall"  where 
people  engaged  in  the  visual  organization  of  information  using  graphical  objects  (Cards)  share  their 
work  via  software  agents  designed  using  psychological  theories  of  cognitive  perception  of  spatial 
object  arrangements. 

[76]  D.  A.  Keim,  “Information  Visualization  and  Visual  Data  Mining,”  IEEE  Transactions  on 
Visualization  and  Computer  Graphics,  vol.  7,  no.  1,  http://fusion.cs.uni- 
magdeburg.de/pubs/TVCG02.pdf,  2002. 

[A  copy  is  attached  as  file:  KeimVisDataMining.pdf] 

Foundational  paper  outlining  the  advantages  of  visualization  for  data  mining  applications,  and 
giving  a  long  list  of  examples. 

[77]  D.  A.  Keim,  F.  Mansmann,  J.  Schneidewind  et  al.,  “Challenges  in  Visual  Data  Analysis,”  in 
Information  Visualization  (IV  2006),  IEEE,  http://www.inf.uni- 
konstanz.de/cgip/bib/files/KeMaSc06.pdf,  2006,  pp.  9-16. 

[A  copy  is  attached  as  file:  KeimChallenges.pdf] 

Provides  an  overview  of  visual  analytics,  its  scope  and  concepts,  and  details  the  most  important 
technical  research  challenges  in  the  field. 
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[78]  J.  Kielman,  “A  Message  from  the  Department  of  Homeland  Security,”  VAC  Views,  pp.  2,  February 
2007,  2007. 

[A  copy  is  attached  as  file:  VACViews02_feb07.pdf] 

A  seminal  call-to-arms  from  a  key  client  in  the  United  States.  Kielman  lays  out  the  challenge  to 
deliver  “designer  information.  ” 

[79]  J.  Kohlhammer,  and  D.  Keim,  "White  paper:  Visual  Analytics  in  Europe  -  Mastering  the  Information 
Age  http://www.igd.fraunhofer.de/igd-a3/downloads/VA/Whitenaper  Visual  Analytics.pdf.  2007. 
[A  copy  is  attached  as  file:  Kohlhammer_Whitepaper.pdf] 

This  whitepaper  indicates  that  there  are  no  European  initiatives  in  Visual  Analytics,  and  suggests 
that  such  initiatives  be  undertaken. 

[80]  G.  Lawton,  “Users  Take  a  Close  Look  at  Visual  Analytics,”  Technology  News,  February  2009,  2009. 
[A  copy  is  attached  as  file:  LawtonUsersCloserLook.pdf] 

This  article  is  of  interest  because  it  is  one  of  the  few  that  references  European  initiative  in  visual 
analytics. 

[81]  R.  Lecocq,  "Knowledge  Mapping  in  Emergency  Operations,"  DRDC  Valcartier,  2009. 

[A  copy  is  attached  as  file:  KMapKnowledge_mappinginEmergOps_FinalPaper6b.ppt] 

KMapper  provides  a  navigable  visualization  of  an  extended  information  space,  including  human 
resources,  knowledge  assets,  procedural  information,  and  conceptual  information.  These  slides 
provide  a  recent  update  on  KMapper  development,  and  are  rich  in  screen  grabs  showing  the 
visualization  strategy’. 

[82]  H.  Luo,  J.  Fan,  D.  A.  Keim  et  al.,  “Personalized  News  Video  Recommendation,”  in  Proceedings  of 
the  15th  Internation  Multimedia  Modeling  Conference 
http://viscenter.uncc.edu/TechnicalReports/CVC-UNCC-09-18.pdf,  2009. 

[A  copy  is  attached  as  file:  LuoNewsVideo.pdf] 

Analyses  large-scale  collections  of  multi-model  news  videos  to  provide  interactive  navigation  and 
exploration  at  the  topic  level.  Users  can  build  up  their  mental  models  of  news  needs  precisely  and 
formulate  their  queries  easily  by  directly  selecting  the  visible  news  topics  on  the  topic  network. 

[83]  R.  Maciejewski,  S.  Rudolph,  G.  Tebbetts  et  al.,  “Contextualizing  Syndromic  Hotspots  -  A  Visual 
Analytics  Approach,”  in  Geospatial  Visual  Analytics,  Uta, 
http://www.geoanalvtics.net/GeoVisualAnalytics08/a03.pdf,  2008. 

[A  copy  is  attached  as  file:  Maciejewski_Syndromic_Hotspots.pdf] 

This  visualization  of  spatially-distributed  features  ("syndromic  hot-spot"  in  this  example,  but  could  be 
"gale"  in  the  RMP)  and  displaying  them  as  changing  in  time,  may  be  relevant  to  the  time-varying 
situational  picture. 

[84]  F.  Mansmann,  L.  Meier,  and  D.  A.  Keim,  “Visualization  of  Host  Behavior  for  Network  Security,”  in 
Workshop  on  Visualization  for  Computer  Security,  Sacramento,  IEEE,  http://infovis.uni- 
konstanz.de/papers/2008/vizsec2007.pdf,  2007. 

[A  copy  is  attached  as  file:  Mansmann_HostNetSecurity.pdf] 
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[85]  M.  Matthews,  L.  Rehak,  J.  Famewo  et  al.,  "Evaluation  of  New  Visualization  Approaches  for 
Representing  Uncertainty  in  the  Recognized  Maritime  Picture,"  DRDC  Atlantic  CR  2008-177,  DRDC 
Atlantic,  2008. 

[A  copy  is  attached  as  file:  MatthewsUncertainty.pdf] 

This  DRDC  study  focuses  on  representing  uncertainty  and  swath  coverage  in  the  Maritime  Picture.  It 
includes  a  literature  review,  user  interface  brainstorming,  protoyping  of  some  designs,  and  tests  with 
CF  analysts. 

[86]  R.  May,  P.  K.  Arya,  D.  A.  Bowman  et  al.,  “Challenges  to  Applying  Virtual  Reality  Technology  and 
Techniques  to  Visual  Analytics,”  in  Virtual  Reality  Conference,  2006,  2006,  pp.  303-306. 

A  panel  discussion  to  introduce  the  domain  of  visual  analytics  to  the  virtual  reality  (VR)  and  artificial 
reality  (AR)  audience  and  explore  how  and  where  VR/AR  research  can  be  adapted  for  use  in  visual 
analytics. 

[87]  S.  Mehta,  S.  Parthasarathy,  and  R.  Machiraju,  “Visual  Exploration  of  Spatio-temporal  Relationships 
for  Scientific  Data,”  in  IEEE  Visual  Analytics  Science  and  Technology  (VAST  06),  ftp://ftp.cse.ohio- 
state.edu/pub/tech-report/2006/TR33.pdf,  2006. 

[A  copy  is  attached  as  file:  MehtaSpatio-Temporal  Relat.pdf] 

Mostly  useful  for  analysis  of periodic  motion.  Not  very  rich  in  either  visualization  or  support  for 
analysis. 

[88]  L.  Meier.  "Visualization  of  Network  Time  Series,"  web  page  at:  http://graphics.uni-konstanz.de/bw- 
fit/pro j ekte/pro j ekt7 .php,  2008. 

[A  copy  is  attached  as  file:  MeierNetworkTimeSeries.mht] 

A  lab  web  page  (in  German)  focused  on  visualizing  network  behaviour  (e.g,  load  profiles,  port 
information,  IP  packet  information,  etc.)  as  a  function  of  time.  The  specific  challenges  are  in  the 
extremely  large  data  sets  and  the  high  degree  of  detail  required. 

[89]  B.  S.  Meiguins,  and  A.  S.  G.  Meiguins,  “Multiple  coordinated  views  supporting  visual  analytics,”  in 
SIGKDD  Workshop  on  Visual  Analytics  and  Knowledge  Discovery:  Integrating  Automated  Analysis 
with  Interactive  Exploration,  Paris,  France,  ACM,  http://www.hiit.fi/vakd09/vakd09meiguins.pdf, 
2009,  pp.  40-45. 

[A  copy  is  attached  as  file:  MeiguinsMultiCoordViews.pdf] 

Uses  their  Java  tool  PRISMA  to  analyse  the  IEEE  VAST  2008  contest  data.  Coordination  between 
views  is  based  on  filters,  colour,  shape,  and  size,  plus  support  for  details-on-demand.  It  appears  that 
the  interface  was  very  customized  for  this  challenge. 

[90]  D.  Mewett,  and  D.  Clark,  “Dynamic  Display  for  Visualising  Surveillance  Coverage  in  Complex 
Terrain,”  in  DSTO  External  Publications,  DSTO,  http://www.siaa.asn.au/get/241 1853305.pdf,  2004. 
[A  copy  is  attached  as  file:  MewettSurvCovg.pdf] 

This  is  one  of  very >  few  papers  that  looks  at  visualization  of  sensor  coverage  patterns  over  time.  The 
authors  implemented  a  solution  in  which  coverage  areas  fade  with  time,  to  indicate  that  the  data  is 
stale. 
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[91]  MIAS.  "Center  for  Multimodal  Information  Access  and  Synthesis  Center,"  web  page  at: 
http://mias.uiuc.edu/,  2009. 

[A  copy  is  attached  as  file:  MiasHomePage.mht] 

Web  page  for  MIAS,  giving  links  to  faculty  and  projects. 

[92]  National  Visualization  and  Analytics  Center.  "NVAC  Call  for  Internships  &  Fellowships,"  2009.  [A 
copy  is  attached  as  file:  NVAC  Call  For  Intems.mht] 

This  web  page  provides  a  good  quick  overview  of  what  NVAC  does. 

[93]  National  Visualization  and  Analytics  Center,  VAC  Views,  nvac.pnl.gov/vacviews,  2009. 

[See file:  VACViews06_may09.pdf] 

VAC  Views  is  the  only  publication  dedicated  to  Visual  Analytics.  It  is  a  magazine  rather  than  a 
scholarly  journal,  but  provides  good  insights  into  activities  around  the  VAC  Consortium. 

[94]  NEVAC,  "North-East  Visualization  and  Analytics  Center  "  available  on-line  at: 
http://www.geovista.psu.edu/NEVAC/,  2009. 

[A  copy  is  attached  as  file:  NEVAC  Penn  State  University.mht] 

[95]  NVAC.  "National  Visualization  and  Analytics  Center  Research  and  Development,"  web  page  at: 
http://nvac.pnl.gov/research.stm,  2009. 

[A  copy  is  attached  as  file:  NvacResearch.mht] 

The  NVAC  continues  to  a  lot  of  in-house  research  and  the  Pacific  Northwest  National  Lab,  as 
summarized  and  linked  here. 

[96]  NVAC.  "Visualization  &  Analytics  Centers  (VAC)  Consortium,"  web  page  at: 
http://nvac.pnl.gov/consortium.stm,  2009. 

[A  copy  is  attached  as  file:  VAC  Consortium. mht] 

The  VA  consortium  includes  both  users  and  producers  of  VA  tools.  It  provides  value  to  its  members 
and  to  DHS,  by  giving  valuable  insight  into  the  activities  of  groups  within  the  NVAC  program  and  by 
influencing  the  direction  future  generations  of  visual  analytics.  This  website  gives  links  to 
information  pages  about  all  previous  VAC  consortium  meetings. 

[97]  Oculus.  "GeoTime:  Efficient  Insight  into  Behaviors  in  Time  and  Space,"  web  page  at: 
http://www.oculusinfo.com/SoftwareProducts/GeoTime.html,  2009. 

[A  copy  is  attached  as  file:  Oculus  -  GeoTime.mht] 

[98]  Pacific  Northwest  National  Laboratory  (PNNL),  "In-Spire  Tips  and  Techniques  "At  a  Glance" 
Version  3.0,"  http ://in-spire.pnl. gov/TipsandT echniques3 O.pdf,  2005. 

[A  copy  is  attached  as  file:  InSpireTips.pdf] 

This  is  an  excellent  introduction  to  how  In-Spire  works,  including  a  careful  description  of  how 
analysis  can  progress  from  one  visualization  to  another.  It  shows  examples  of  all  the  major  screens. 


B-14 


Use,  duplication,  or  disclosure  of  this  document  or  any  of  the  information 
contained  herein  is  subject  to  the  restrictions  on  the  title  page  of  this  document. 


BMDA 


Ref: 

Issue/Revision: 

Date: 


RX-RP-52-8724 


1/0 

OCT.  28,  2009 


[99] 


[100] 


[101] 


[102] 


[103] 


[104] 


[105] 


Pacific  Northwest  National  Laboratory  (PNNL).  "Assessment  Wall,"  web  page  at: 
http://infoviz.pnl.gov/tech  assessment.stm,  2008. 

\A  copy  is  attached  as  file:  Assessment  Wall.mht] 

This  is  designed  to  help  users  more  easily  monitor  and  explore  continually  updating  text  sources  such 
as  news  or  reports,  through  a  high-resolution  touch-screen  interface. 

Pacific  Northwest  National  Laboratory  (PNNL),  "In-Spire  Visual  Document  Analysis,"  Youtube 
available  on-line  at:  http://www.youtube.com/watch?v=7bIRmJlhibQ&feature=channel  page,  2009. 
[A  copy  is  attached  as  file:  Youtube  lnSpire.mht] 

This  Youtube  video  marketing  pitch  is  less  informative  than  the  Starlight  youtube,  bur  still  of  interest 
because  it  shows  the  tool  at  work,  and  gives  ideas  of  possible  applications. 

Pacific  Northwest  National  Laboratory  (PNNL),  "Starlight  Information  Visualization  System," 
YouTube  video  available  on-line  at: 

http://www.youtube.com/watch?v=3rV62mVFMx8&feature=channel  page,  2009. 

[A  copy  is  attached  as  file:  YouTube  -  Starlight.mht] 

Although  this  YouTube  video  only  touches  the  surface  of  what  Starlight  can  do,  it  is  valuable  because 
it  shows  the  dynamics  of  some  of  the  visualization  -  something  that  cannot  be  inferred  from  static 
publications. 

Palgrave  McMillan  Ltd.,  Information  Visualization,  http://www.palgrave- 
ioumals.com/ivs/index.html,  2009. 

This  journal  is  newer  and  less  well-known  than  the  IEEE  journal,  but  is  being  supported  by  leading 
visual  analytics  researchers  such  as  John  Stasco. 

A.  Perer,  and  B.  Shneiderman,  “Integrating  Statistics  and  Visualization  for  Exploratory  Power:  From 
Long-Term  Case  Studies  to  Design  Guidelines,”  IEEE  Computer  Graphics  and  Applications,  pp.  39- 
51,  http://www.computer.org/portal/cms  docs  cga/cga/content/cga  per  3  09.pdf,  2009. 

[A  copy  is  attached  as  file:  Perer_EvaluatingVA_2009.pdf] 

A  useful  magazine  article  in  which  the  links  between  data  mining  and  visual  analytics  are  developed. 

Purdue  University.  "Purdue  University  Regional  Visualization  and  Analytics  Center,"  web  page  at: 
https://engineering.purdue.edu/PURVAC/pdf_fliers/PURVAC01_Overview.pdf,  2009. 

[A  copy  is  attached  as  file:  PURVAC01_Overview.pdf] 

PURVAC  is  researching:  data  integration,  Effective  Analysis  of  Anonymized  Data,  Disaster  Response 
Life  Cycle,  Mobile  Analytics  for  Command  &  Control  and  Emergency  Response,  Personnel  Tracking 
and  Video  Capture  For  Emergency  Response,  Sensor-Driven  Video  Analytics,  Social  Networks, 
Zoonotic  Disease  Spread,  Time-Space  Modeling  and  Analysis  for  Disease  Surveillance,  and  Network 
Flow  Monitoring  and  Forensics 

W.  Renaud,  and  A.  Isenor,  “Situational  Awareness:  Inspiring  Operational  Clarity,”  Canadian  Military ? 
Journal  vol.  10,  no.  4,  Winter,  2009. 

[A  copy  is  attached  as  file:  RenaudSituAware.pdf] 
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This  is  Cdr  Renaud's  "manifesto"  for  situational  awareness,  and  thus  a  very  helpful  window  into  CF 
thinking  and  planning  for  Maritime  Awareness. 

[106]  M.  Riveiro,  “Evaluation  of  uncertainty  visualization  techniques  for  information  fusion,”  in 
Information  Fusion,  2007  10th  International  Conference  on,  2007,  pp.  1-8. 

Looks  at  uncertainty  representations  in  a  number  of  fields,  from  graphical  rendering  to  surveillance 
displays,  but  does  not  come  up  with  definitive  recommendations  about  how  it  should  be  done. 

[107]  M.  Riveiro,  G.  Falkman,  T.  Ziemke  et  al.,  “Reasoning  about  anomalies:  a  study  of  the  analytical 
process  of  detecting  and  identifying  anomalous  behavior  in  maritime  traffic  data,”  in  Proc.  of  SPIE 
Vol.  7346,  73460A  SPIE, 

http://spie.org//app/program/index.cfm?fuseaction=conferencedetail&confid=869287,  2009. 

Riveiro  reports  on  obsen’ations  of  maritime  surveillance  operators  looking  for  anomalies, and 
outlines  a  3-step  surveillance  process,  which  she  compares  to  "overview,  zoom,  details  on  demand, " 
and  to  the  foraging  loop.  A  key  observation  is  that  there  is  always  a  "wait"  step,  while  the  analysts 
clarify  what  the  motion  is,  etc. 

[108]  M.  Riveiro,  G.  Falkman,  T.  Ziemke  et  al.,  “VISAD:  an  interactive  and  visual  analytical  tool  for  the 
detection  of  behavioral  anomalies  in  maritime  traffic  data,”  in  Visual  Analytics  for  Elomeland 
Defense  and  Security,  SPIE, 

http://spiedigitallibrarv.aip.org/getpdf/servlet/GetPDFServlet?filetype=pdf&id=PSISDG0073460000 

0 173460700000 1  &idtvpe=cvips&prog=normal,  2009. 

[A  copy  is  attached  as  file:  RiveiroVisad.pdf] 

This  is  a  maritime  anomaly  detection  system  that  preprocesses  the  maritime  reports  in  order  to 
highlight  potential  anomalies,  then  offers  the  operator  a  screen  that  describes  the  ship  (though  it 
gives  no  obvious  description  of  why  it  was  flagged  as  an  anomaly).  It  is  not  clear  from  the  paper 
whether  VISAD  helps  the  operator  to  build  a  visual  analysis  of  the  vessels  -  VISAD  may  just  be  a 
visualization  system.. 

[109]  M.  Riveiro,  G.  o.  Falkman,  and  T.  Ziemke,  “Visual  Analytics  for  the  Detection  of  Anomalous 
Maritime  Behavior,”  in  12th  International  Conference  Information  Visualisation,  IEEE,  2008. 

[A  copy  is  attached  as  file:  Riveiro_VaAnomMaritime.pdf] 

Describes  a  system  that  supports  the  acquisition  of  situation  awareness  with  user  involvement  in  the 
anomaly  detection  process  using  two  layers  of  interactive  visualizations.  An  interactive  data  mining 
module  supports  the  insertion  of  the  user ’s  knowledge  and  experience.  The  visual  language  used  in 
this  prototype  is  in  some  cases  a  poor  match  to  the  domain. 

[110]  J.  C.  Roberts,  “State  of  the  Art:  Coordinated  &  Multiple  Views  in  Exploratory  Visualization,”  in 
Coordinated  &  Multiple  Views  in  Exploratory  Visualization  (CMV  07)  Zurich,  IEEE, 
http://people.cs.ubc.ca/~tmm/courses/533/readings/cmvsurvey.pdf,  2007. 

[A  copy  is  attached  as  file:  RobertsCmvSurvey.pdf] 

Gives  an  over\>iew  of  Coordinated  and  Multiple  Views  research,  which  is  a  core  component  of  many 
Visual  Analytics  products. 
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[111] 

[112] 

[113] 

[114] 

[115] 

[116] 

[117] 

[118] 

[119] 


A.  C.  Robinson,  “Collaborative  synthesis  of  visual  analytic  results,”  in  Visual  Analytics  Science  and 
Technology,  2008.  VAST  '08.  IEEE  Symposium  on,  2008,  pp.  67-74. 

Robinson  ran  experiments  to  watch  how  people  collaborate,  and  then  built  an  architect  to  support  the 
various  approaches  that  they  used. 

T.  Rowland,  "Spies  Look  to  Second  Life  and  Think  Time  Machine!,"  Got  Game, 
news.gotgame.com/.. ./1281/,  2008. 

J.  Roy,  “Anomaly  detection  in  the  maritime  domain,”  in  Optics  and  Photonics  in  Global  Elomeland 
Security  IV,  SPIE,  2008. 

SAIC.  "SAIC:  Future  Point  Systems  and  the  Starlight  Visual  Information  System,"  web  page  at: 
http://www.saic.com/contractcenter/gsa-it/futurepoint.html,  2009. 

Mainly  of  interest  because  of  the  pricing  information  on  Starlight. 

San  Diego  State  University,  "3D  Touch  Table  in  ESRI  2005  User  Conference,"  available  on-line  at: 
http://www.youtube.com/watch?v=W3dz2xpCJVU,  2005. 

[A  copy  is  attached  as  file:  3DTouchTable.avi] 

This  is  a  video  of  the  3D  touch  table  in  action. 

T.  Schreck,  J.  Bernard,  T.  Tekusova  et  al.,  “Visual  cluster  analysis  of  trajectory  data  with  interactive 
Kohonen  maps,”  in  IEEE  Symposium  on  Visual  Analytics  Science  and  Technology  08,  Columbus, 
IEEE,  http://www.gris.informatik.tu-darmstadt.de/~tschreck/papers/vast08.pdf,  2008. 

[A  copy  is  attached  as  file:  Schreck  Trajectories.pdf] 

To  simplify  the  RMP  it  may  be  helpful  to  find  common  patterns  in  the  shapes  of  ship  tracks,  using  an 
automated  tool  to  indicate  whether  two  tracks  have  a  similar  shape.  This  paper  looks  at  using  a  self¬ 
organizing  map  for  that  purpose.  This  research  uses  Visual  Analytics  to  help  guide  the  self- 
organizing  map  to  create  the  tool. 

Visualization,  Analytics  &  Spatial  Decision  Support  2006. 

No  relevant  papers  for  Maritime  Awareness  were  found  in  this  workshop. 

B.  Shneiderman,  “The  eyes  have  it:  a  task  by  datatype  taxonomy  for  information  visualization,”  in 
IEEE  Symposium  on  Visual  Languages,  IEEE  Computer  Society  Press, 
hhttps://eprints.kfupm.edu.sa/69898/l/69898.pdf,  1996,  pp.  336-343. 

[A  copy  is  attached  as  file:  SchneidermanEyesHaveIt.pdf] 

This  is  a  seminal  paper  by  Schneiderman  laying  out  a  very  practical  taxonomy  of  visualization 
strategies  and  domains.  Not  explicitly  Visual  Analytics,  but  an  important  foundation  for  it. 

Sifer,  “User  Interfaces  for  the  Exploration  of  Elierarchical  Multi-dimensional  Data,”  in  IEEE 
Symposium  On  Visual  Analytics  And  Technology,  Baltimore,  IEEE, 

http://ro.uow.edu.au/cgi/viewcontent.cgi7articleM 304&context=commpapers,  2006,  pp.  175-182. 

[A  copy  is  attached  as  file:  SiferHier.pdf] 
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Defines  metrics  to  compare  interfaces  for  interactive  exploration  of  hierarchical  multi-dimensional 
data ,  and  uses  those  metrics  to  evaluate  various  visualization  strategies 

[120]  Sofrelog.  "Sofrelog  the  leader  in  radar  processing  technology,"  web  page  at: 
http://www.sofrelog.com/main  index.htm,  2008. 

[A  copy  is  attached  as  fde:  Sofrelog.mht] 

[121]  Visual  Analytics  for  Homeland  Defense  and  Security  (Conference  7346)  in  Defense,  Security,  & 
Sensing,  http://spie.org//app/program/index.cfm?fuseaction=conferencedetail&confid=869287.  2009. 
[See file:  SpieConferenceDetail.mht] 

Only  one  relevant  paper  was  found  from  this  small  session  in  the  annual  SPIE  conference. 

[122]  Stanford.  "Stanford  vis  Group,"  web  page  at:  http://vis.stanford.edu/,  2009. 

[A  copy  is  attached  as  file:  Visualization  at  Stanford  University.mht] 

This  is  a  page  of  links  to  the  Stanford  visualization  and  visual  analytics  pages. 

[123]  J.  Stasco.  "Information  Interfaces  Group,"  web  page  at:  http://www.cc.gatech.edu/gvu/ii/,  2009. 

[A  copy  is  attached  as  file:  GeorgiaTech.mht] 

This  is  the  home  page  for  John  Stasco' s  lab  at  Georgia  Tech. 

[124]  J.  Stasko,  C.  Gorg,  and  Z.  Liu,  “Jigsaw:  supporting  investigative  analysis  through  interactive 
visualization,”  Information  Visualization,  vol.  7,  pp.  118-132,  http://www.palgrave- 
ioumals.com/ivs/ioumal/v7/n2/pdf/9500 180a.pdf,  2008. 

[A  copy  is  attached  as  file:  StascoJigsaw.pdf] 

A  good  journal  article  describing  Jigsaw,  including  a  frank  discussion  of  some  of  its  limitations  (e.g. 
Jigsaw  does  not  provide  overviews  that  help  an  analyst  find  something  interesting  to  start  looking  at. 

[125]  J.  Stasko,  C.  Gorg,  Z.  Liu  et  al.,  "Jigsaw  -  Visual  Analytics,"  Youtube  available  on-line  at: 
http://www.voutube.com/user/cagoe,  2008. 

[A  copy  is  attached  as  file:  jigsaw_video.avi] 

[126]  B.  Suh,  E.  H.  Chi,  B.  A.  Pendleton  et  al.,  “Us  vs.  Them:  Understanding  Social  Dynamics  in 
Wikipedia  with  Revert  Graph  Visualizations,”  in  IEEE  Symposium  on  Visual  Analytics  Science  and 
Technology  2007  Sacramento,  IEEE,  http://www.citeulike.org/user/vulpeto/article/2359769,  2007. 

Show  how  visualization  and  visual  analytics  can  provide  important  insights  into  Wiki-type 
collaborative  knowledge  bases  (such  as  Intellipedia  for  example). 

[127]  Thetus,  "The  Thetus  Intelligence  Fusion  Platform,"  Thetus  Inc., 
http://fusion.thetus.com/docs/Thetus  Intel  Fusion.pdf,  2007. 

[A  copy  is  attached  as  file:  Thetus_Intel_Fusion.pdf] 

The  Thetus  Intelligence  Fusion  Platform  seems  to  resemble  Analyst's  Notebook,  but  gives  more 
visibility  to  an  underlying  "Knowledge  Base"  in  the  form  of  an  ontology >  that  acts  as  a  repository  for 
the  emerging  ", story > "  that  is  built  up  by  the  analyst. 
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[128]  Thetus.  "Thetus  Knowledge  Services,"  web  page  at:  http://www.thetus.com/index.html,  2008. 

[A  copy  is  attached  as  file:  Thetus  Thetus  Knowledge  Services.mht] 

Thetus  started  out  hoping  to  be  like  i2,  only  with  more  intelligence;  more  real  fusion.  Now  they  seem 
to  have  given  up  on  marketing  to  the  police  and  military >.  They  are  big  on  semantic  modelling  and 
tracking  meaning  and  provenance  of  information. 

[129]  J.  Thomas,  “Taxonomy  for  Visual  Analytics:  Seeking  Feedback,”  VAC  Views,  pp.  6-7,  May  2009, 
2009. 

[A  copy  is  attached  as  file:  VACViews06_may09.pdf] 

This  is  a  typology >  of  Visual  Analytics  domains,  goals,  data  types,  and  technologies,  offered  by  Jim 
Thomas  as  a  discussion  point. 

[130]  J.  J.  Thomas,  "Visual  Analytics  Techniques  that  Enable  Knowledge  Discovery:  Detect  the  Expected 
and  Discover  the  Unexpected,"  Director,  National  Visualization  and  Analytics  Center, 
http://www.hiit.fi/vakd09/vakd09thomas-slides.pdf,  2009. 

[A  copy  is  attached  as  file:  ThomasVakd09-slidcs.pdf] 

Slides  that  provide  a  brief  over-view  of  Visual  Analytics  from  the  guru  himself.  The  first  part  of  the 
presentation  is  a  useful  discussion  of  what  is  (and  is  not)  Visual  Analytics. 

[131]  Author  ed.eds.,  “Illuminating  the  Path,”  IEEE,  http://nvac.pnl.gOv/agenda.stm#book,  2005,  p.pp. 
Pages. 

[See  file:  IlluminatingThePath_Thomas.pdf] 

This  is  the  key  foundational  reference  that  originally  set  the  agenda  for  Visual  Analytics  development, 
and  continues  to  be  very  influential. 

[132]  University  of  Konstanz.  "VisMaster,"  web  page  at:  http://www.vismaster.eu/node/2,  2008. 

[A  copy  is  attached  as  file:  VisMaster.mht] 

This  is  the  homepage  for  VisMaster,  a  European  attempt  to  coordinate  some  visual  analytics  research 
initiatives.  The  website  was  created  in  October  2008,  but  the  last  news  update  on  the  page  was 
March  12,  2009.  A  scan  of  the  website  links  for  5  of  the  17  "VisMaster  Partner"  agencies  revealed 
none  of  the  websites  carried  the  words  "VisMaster"  or  "Analytics. " 

[133]  University  of  Konstanz.  "Chair  for  Databases,  Data  Analysis  and  Visualization  "  web  page  at: 
http ://www. informatik. uni-konstanz.de/ en/ arbeitsgruppen/infovis/,  2009. 

[A  copy  is  attached  as  file:  UKonstanz  Data  Analysis  and  Visualization.mht] 

This  lab  (under  Daniel  Keim)  focuses  on  techniques  for  interactive  mass  data  analysis  for  very’  large 
multidimensional  and  geographic  data  sets,  clustering  methods  and  index  structures  for  high- 
dimensional  spaces,  and  multimedia  similarity  search  (2D  and  3D).  Note  that  the  home  page  never 
mentions  visual  analytics,  so  this  may  just  be  visualization. 

[134]  University  of  Washington.  "Pacific  Rim  Visualization  and  Analytics  Center  (PARVAC),"  web  page 
at:  http://parvac.washington.edu/index.php,  2009. 

[A  copy  is  attached  as  file:  PARVAC  for  Area  Maritime  Security.mht] 
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PARVAC  plans  to  develop:  a  map  of  the  sense-making,  decision-making  and  coordination  processes 
of  the  diverse  Area  Maritime  Security  community,  both  during  non-crisis  and  crisis  situations;  an 
evolving  map  of  technology  components  that  support  these  AMS  processes,  such  as  command  and 
control  (C2)  systems,  analysis  tools,  and  databases;  and  a  visual  representation  of  how  existing 
technology >  and  possible  analytic  and  coordination  enhancements  map  to  these  regional  safety  and 
security  processes. 

[135]  UNOSAT.  "Pirate  Attack  Density  in  the  Gulf  of  Aden  "  web  page  at: 
http://unosat.web.cem.ch/unosat/freeproducts/somalia/Piracv/lINOSAT  SO  PirateDensity  Nov08 

Highres  vl.pdf,  2008. 

[A  copy  is  attached  as  file:  UNOSAT_PirateDensity_Nov08.pdf] 

Illustrates  a  number  of  visualization  strategies  for  visual  data  mining  of  maritime  events.  (UNOSAT 
is  UN  Institute  for  Training  and  Research  (UNITAR)  Operational  Satellite  Applications  Programme) 

[136]  R.  Vernik,  G.  S.  Von  Itzstein,  and  A.  Bouchard,  “Imago:  An  integrated  prototyping,  evaluation  and 
transitioning  environment  for  information  visualisation,”  in  Information  Visualization,  2007.  IV  '07. 

1 1th  International  Conference,  2007,  pp.  17-22. 

[A  copy  is  attached  as  file:  VernikImago.pdf] 

Imago  is  a  distributed  environment  that  supports  visualization,  and  thus  a  candidate  for  hosting 
collaborative  visual  analytics. 

[137]  Visual  Analytics  Inc.  "Home  Page,"  web  page  at:  http://www.visuala.com/index.cfm,  2009. 

[A  copy  is  attached  as  file:  Visual  Analytics  Inc.mht] 

Visual  Analytics  Inc.  has  literally  had  the  copyright  on  the  term  "Visual  Analytics"  since  1998.  They 
consult  to  Homeland  Defence  and  security  agencies,  but  a  lot  of  their  business  is  with  financial  and 
market  analytics.  They  sponsor  VisuaLinks  and  Digital  Information  Gateway  tools. 

[138]  Visual  Analytics  Inc.  "VisuaLinks  -  The  Pattern  Discovery  Software,"  web  page  at: 
http://www.visualinks.com/,  2009. 

Visualinks  seems  to  be  similar  to  12,  accessing  databases,  searching  for  implicit  relationships,  and 
displaying  them  visually. 

[139]  C.  Weaver,  “Is  Coordination  a  Means  to  Collaboration?,”  in  International  Conference  on  Coordinated 
&  Multiple  Views  in  Exploratory  Visualization  (CMV  07),  Zurich,  SZ,  IEEE, 
http://www.cs.ou.edu/~weaver/academic/publications/weaver-2007c.pdf,  2007. 

Explores  how  collaboration  can  be  achieved  by  giving  separate  users  coordinated  and  linked  views  of 
the  analysis  space,  and  giving  managers  tools  to  "metavisualize"  the  data. 
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[140] 


[141] 


[142] 


[143] 


[144] 


L.  Wilkinson,  A.  Anand,  and  R.  Grossman,  “High-Dimensional  Visual  Analytics:  Interactive 
Exploration  Guided  by  Pairwise  Views  of  Point  Distributions,”  IEEE  Transactions  in  Visualization 
and  Computer  Graphics,  vol.  12,  no.  6,  pp.  1363-1372,  Nov/Dec  2006, 
http://www.cs.uic.edu/~wilkinson/Publications/sorting.pdf,  2006. 

[A  copy  is  attached  as  fde:  WilkinsonHighDimensionalVA.pdf] 

A  detailed  look  at  finding  patterns  in  very  high-dimensional  data,  using  statistical  transformations  to 
re-project  the  data  and  thus  reveal  the  patterns,  and  plots  that  link  large  numbers  of  two-dimensional 
views  (usually  scatter  plots)  of  the  data. 

N.  Willems,  H.  van  de  Wetering,  J.  van  Wijk  et  al.,  “Visualization  of  vessel  trajectories  for  maritime 
safety  and  security  systems,”  in  IEEE  Symposium  on  Visual  Analytics  Science  and  Technology 
(VAST  08),  Columbus,  Ohio,  IEEE, 

http://www.esi.nl/proiects/poseidon/publications/InteractivePosterVisualization  Willems.pdf,  2008. 

[A  copy  is  attached  as  file:  Willems_Vis_Traject2.pdf  and  Willems_Vis_Traject.pdf] 

Looks  at  a  very  specific  graphical  strategy  for  visualizing  track  patterns  in  a  very  busy  part  of  the 
North  Sea,  using  AIS  data.  Tracks  are  interpolated  and  convolved  with  a  kernel  in  such  a  way  that 
the  accumulated  view  represent  ship  density,  rather  than  number  of  tracks.  (10  slow-moving  ships 
are  more  dense  than  15  fast-moving  ships). 

N.  Willems,  H.  van  de  Wetering,  J.  van  Wijk  et  al.,  “Visualization  of  vessel  movements  ”  in  1 1th 
Eurographics/IEEE-VGTC  Symposium  on  Visualization  (Computer  Graphics  Forum;  Proceedings  of 
EuroVis  2009),  2009  IEEE,  http://www.win.tue.nl/~cwillems/public/eurovis09.pdf,  2009. 

[A  copy  is  attached  as  file:  Willems_Vis_Traject3.pdf] 

This  paper  follows-up  publications  by  Willems  in  2008,  showing  how  to  visualize  aggregated  vessel 
trajectories.  The  result  is  a  nice,  clear  indication  of  both  individual  tracks  and  time-averaged  track. 

W.  Wright,  and  T.  Kapler,  “Visualization  of  Blue  Forces  Using  Blobology,”  in  Command  and 
Control  Research  and  Technology  Symposium,, 

http://www.dodccrp.org/events/2002  CCRTS/Tracks/pdf/029.PDF,  2002. 

[A  copy  is  attached  as  file:  wrightBlobology2002.pdf] 

Blobology  provides  a  way  to  represent  a  distributed  group  of  individuals,  including  the  time- 
evolution  of  the  shape  and  location  of  that  group. 

Z.  Xie,  S.  Huang,  M.  O.  Ward  et  al.,  “Exploratory  Visualization  of  Multivariate  Data  with  Variable 
Quality,”  in  Visual  Analytics  Science  and  Technology  (VAST  06),  IEEE, 
http://davis.wpi.edu/~xmdv/docs/vast06  dataqua.pdf,  2006. 

[A  copy  is  attached  as  file:  XieVisualizeQuality.pdf] 

An  analysis  of  various  ways  to  visually  reveal  errors  or  data  quality  problems  in  individual  numbers, 
whole  records,  or  data  sources. 
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[145]  F.  Yang,  H.  Goodell,  R.  Pickett  et  al.,  “Data  Exploration  Combining  Kinetic  and  Static  Visualization 
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